matplotlib - then and now

[darkblueb]

many packages in UbuntuGIS-unstable refer to matplotlib

python-matplotlib (1.1.1~rc1+git20120423-0ubuntu1 Ubuntu:12.04/precise [i386])

but, iPython notebook - SciPy - matplotlib is on version 1.3

there have been substantial improvements to matplotlib and iPython Notebook since Ubuntu 12.04 was finished.

[darkblueb]

currently, the qgis package in Unstable refers to : python-matplotlib (1.1.1~rc1+git20120423-0ubuntu1 Ubuntu:12.04/precise [i386])

also note: python-gdal (1.10.0-1~precise1 ubuntugis-unstable:12.04/precise [i386])

[johanvdw]

FYI matplotlib will be updated to version 1.3 in Trusty (ubuntu 13.04) which we will use for the next live dvd. ​https://launchpad.net/ubuntu/+source/matplotlib Any effort done is therefore only useful for this live dvd.

[kalxas]

I agree that this should be solved in 8.0 unless someone can provide a fast backport from Trusty to Precise.

[darkblueb]

Today the iPython team responded with links to two repos.. One is straight Ubuntu 12.04, the other is all Debian (Neuroscience).

I added the jtaylor PPA to an installed recent alpha build, and got this result

user@live-nightly79a:~$ sudo apt-get install ipython
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  fonts-font-awesome ipython-notebook ipython-notebook-common ipython-qtconsole libjs-highlight libjs-jquery-ui
  libjs-marked
Suggested packages:
  ipython-doc libjs-jquery-ui-docs
Recommended packages:
  javascript-common
The following NEW packages will be installed:
  fonts-font-awesome libjs-highlight libjs-jquery-ui libjs-marked
The following packages will be upgraded:
  ipython ipython-notebook ipython-notebook-common ipython-qtconsole
4 upgraded, 4 newly installed, 0 to remove and 22 not upgraded.
Need to get 2,660 kB of archives.
After this operation, 6,420 kB of additional disk space will be used.
Do you want to continue [Y/n]? 

Get:1 http://mirror.uoregon.edu/ubuntu/ precise/universe libjs-jquery-ui all 1.8.ooops.18+dfsg-1 [322 kB]
Get:2 http://ppa.launchpad.net/jtaylor/ipython/ubuntu/ precise/main fonts-font-awesome all 4.0.0~dfsg-1~ubuntu12.04.1~ppa1 [270 kB]
Get:3 http://ppa.launchpad.net/jtaylor/ipython/ubuntu/ precise/main ipython-qtconsole all 1.1.0-1~ubuntu12.04.1~ppa1 [81.9 kB]
Get:4 http://ppa.launchpad.net/jtaylor/ipython/ubuntu/ precise/main libjs-highlight all 7.4+ds-1~ubuntu12.04.1~ppa2 [140 kB]
Get:5 http://ppa.launchpad.net/jtaylor/ipython/ubuntu/ precise/main libjs-marked all 0.3.0+dfsg-1~ubuntu12.04.1~ppa1 [13.0 kB]
Get:6 http://ppa.launchpad.net/jtaylor/ipython/ubuntu/ precise/main ipython-notebook all 1.1.0-1~ubuntu12.04.1~ppa1 [32.2 kB]
Get:7 http://ppa.launchpad.net/jtaylor/ipython/ubuntu/ precise/main ipython-notebook-common all 1.1.0-1~ubuntu12.04.1~ppa1 [1,019 kB]
Get:8 http://ppa.launchpad.net/jtaylor/ipython/ubuntu/ precise/main ipython all 1.1.0-1~ubuntu12.04.1~ppa1 [782 kB]
Fetched 2,660 kB in 4s (577 kB/s)
Selecting previously unselected package fonts-font-awesome.
(Reading database ... 258322 files and directories currently installed.)
Unpacking fonts-font-awesome (from .../fonts-font-awesome_4.0.0~dfsg-1~ubuntu12.04.1~ppa1_all.deb) ...
Preparing to replace ipython-qtconsole 0.13.2-1~ubuntu12.04.1 (using .../ipython-qtconsole_1.1.0-1~ubuntu12.04.1~ppa1_all.deb) ...
Unpacking replacement ipython-qtconsole ...
Selecting previously unselected package libjs-highlight.
Unpacking libjs-highlight (from .../libjs-highlight_7.4+ds-1~ubuntu12.04.1~ppa2_all.deb) ...
Selecting previously unselected package libjs-jquery-ui.
Unpacking libjs-jquery-ui (from .../libjs-jquery-ui_1.8.ooops.18+dfsg-1_all.deb) ...
Selecting previously unselected package libjs-marked.
Unpacking libjs-marked (from .../libjs-marked_0.3.0+dfsg-1~ubuntu12.04.1~ppa1_all.deb) ...
Preparing to replace ipython-notebook 0.13.2-1~ubuntu12.04.1 (using .../ipython-notebook_1.1.0-1~ubuntu12.04.1~ppa1_all.deb) ...
Unpacking replacement ipython-notebook ...
Preparing to replace ipython-notebook-common 0.13.2-1~ubuntu12.04.1 (using .../ipython-notebook-common_1.1.0-1~ubuntu12.04.1~ppa1_all.deb) ...
Unpacking replacement ipython-notebook-common ...
Preparing to replace ipython 0.13.2-1~ubuntu12.04.1 (using .../ipython_1.1.0-1~ubuntu12.04.1~ppa1_all.deb) ...
Unpacking replacement ipython ...
Processing triggers for fontconfig ...
Processing triggers for desktop-file-utils ...
Processing triggers for hicolor-icon-theme ...
Processing triggers for man-db ...
Setting up fonts-font-awesome (4.0.0~dfsg-1~ubuntu12.04.1~ppa1) ...
Setting up ipython (1.1.0-1~ubuntu12.04.1~ppa1) ...
Setting up ipython-qtconsole (1.1.0-1~ubuntu12.04.1~ppa1) ...
Setting up libjs-highlight (7.4+ds-1~ubuntu12.04.1~ppa2) ...
Setting up libjs-jquery-ui (1.8.ooops.18+dfsg-1) ...
Setting up libjs-marked (0.3.0+dfsg-1~ubuntu12.04.1~ppa1) ...
Setting up ipython-notebook-common (1.1.0-1~ubuntu12.04.1~ppa1) ...
Setting up ipython-notebook (1.1.0-1~ubuntu12.04.1~ppa1) ...

[darkblueb]

testing iPython and iPython notebook, things are good. The "pylab" side is not so good however. It seems that matplotlib should get an update (0.12 -> 0.13).

Using pip install, shows :

sudo pip install -U matplotlib
...
Downloading matplotlib-1.3.1.tar.gz (42.2Mb): 42.2Mb downloaded
...
The required version of distribute (>=0.6.28) is not available...
(Currently using distribute 0.6.24dev-r0 (/usr/lib/python2.7/dist-packages))
...

so, executed the following:

user@live-nightly79a:~$ sudo easy_install -U distribute
user@live-nightly79a:~$ sudo apt-get install python-dev
user@live-nightly79a:~$ sudo pip install -U matplotlib
...
Successfully installed matplotlib numpy python-dateutil tornado pyparsing nose six backports.ssl-match-hostname
Cleaning up...

[darkblueb]

diff of the python 2.7 environment, pre- and post-install of iPython 1.1 et al

diff live79a_post_ipy.txt ../shared/freeze_pre_ipy.txt 
1a2
> apt-clone==0.2.1
5d5
< backports.ssl-match-hostname==3.4.0.2
19c19
< distribute==0.7.3
---
> distribute==0.6.24dev-r0
37a38
> ## FIXME: could not find svn URL in dependency_links for this package:
49c50
< ipython==1.1.0
---
> ipython==0.13.2
63c64
< matplotlib==1.3.1
---
> matplotlib==1.2.0
68,69c69,70
< nose==1.3.0
< numpy==1.8.0
---
> nose==1.1.2
> numpy==1.6.1
90a92
> PyICU==1.3
94c96
< pyparsing==2.0.1
---
> pyparsing==1.5.2
103c105
< python-dateutil==2.2
---
> python-dateutil==1.5
120c122
< six==1.5.2
---
> six==1.1.0
126c128
< tornado==3.2
---
> tornado==2.1

[darkblueb]

I have now checked QGis, both pre- and post-iPython 1.1, and the same, identical loading error occurs for fTools and gdalTools.

[kalxas]

I would prefer to have all those applications pulled from deb files, as in IPython case.

[kalxas]

darkblueb, new build is up to test: ​http://aiolos.survey.ntua.gr/gisvm/dev/osgeo-live-nightly-build11005-i386.iso ​http://aiolos.survey.ntua.gr/gisvm/dev/osgeo-live-nightly-build11005-i386-log.tar.gz

[kalxas]

As I suspected, we now have python conflicts:

Iris is bringing in "pip install matplotlib==1.2.0" while IPython uses: "pip install --upgrade matplotlib"

Same with numpy which is now built from scratch (gcc) 2 times in the iso creation process, once for IPython and once for Iris.

[darkblueb]

I suggest we turn off Iris, unless Iris can provide an update.

[wildintellect]

I disagree, IRIS being an actual geospatial library with an overview written just needs some more work and clarification. I can see that there is a newer version 1.6 (7.0 had 1.4). What's not clear is if the matplotlib can use a later version. Most deps listed on ​http://scitools.org.uk/iris/docs/latest/installing.html say "or later" but not all of them do. It is listed in the optional section, so I think modifying IRIS install to allow newer should work.

[kalxas]

wildintellect +1

I propose we test latest matplotlib and if Iris doesn't work we revert to 1.2.0 until we move to trusty in 8.0

[darkblueb]

I am seeking ways to prove that the python environment in the current build is stable via testing and analysis.

One test:

• install 12.04.3
• svn co gisvm/trunk
• install selected setup.sh, install_ipython.sh, install_iris.sh
• take a VM snapshot
• sudo apt-get install python-matplotlib
• examine the environment carefully...

[darkblueb]

Please note that there is a matplotlib v1.2 installable via package from ppa:drescampbell/ppa. Using that PPA on a base 12.04.3 plus gisvm setup.sh, shows the following packages to-be-installed (see screenshot)

[darkblueb]

new live7.9 issues, please use a new ticket