Sunday, July 31, 2011

matplotlib on OS X Lion--revised

In the previous post, I repeated an installation of matplotlib (now for OS X Lion) following the same method that I've linked to a number of times on the blog. It worked this time as well, but lead me to consideration of other possible methods. I found one on the web here. So that's what this post is about.

BTW, whatever you do, do not follow the instructions that the matplotlib developers provide. You do not need another Python, including MacPython or the Endthought distribution or anything else..

To recap, matplotlib lists dependencies (in the make.osx file) of: libpng, libfreetype, and zlib. I read somewhere on the matplotlib site today (but can't find the link now) that zlib is not a required dependency. These days, the other two are actually provided by Apple:


> ls -al /usr/X11/lib/libpng*
-rwxr-xr-x 1 root wheel 296864 Jul 29 16:11 /usr/X11/lib/libpng.3.dylib
lrwxr-xr-x 1 root wheel 14 Jul 29 16:11 /usr/X11/lib/libpng.dylib -> libpng15.dylib
-rwxr-xr-x 1 root wheel 294160 Jul 29 16:11 /usr/X11/lib/libpng12.0.dylib
-rwxr-xr-x 1 root wheel 305008 Jul 29 16:11 /usr/X11/lib/libpng15.15.dylib
lrwxr-xr-x 1 root wheel 17 Jul 29 16:11 /usr/X11/lib/libpng15.dylib -> libpng15.15.dylib

> ls -al /usr/X11/lib/libfreetype*
-rwxr-xr-x 1 root wheel 1060416 Jul 29 16:11 /usr/X11/lib/libfreetype.6.dylib
lrwxr-xr-x 1 root wheel 19 Jul 29 16:11 /usr/X11/lib/libfreetype.dylib -> libfreetype.6.dylib


I'm not sure at the moment whether X11 came with Xcode (as it used to) or was present in the original Lion install.

In any case, I spent an hour or two trying to figure out how to use the build commands from the makefile that comes with matplotlib, but point at these libraries. In the process I found what appears to be a bug:


/usr/X11/include/png.h:666: error: forward declaration of ‘struct png_info_def’


(and see Prashant's answer here), but we're going to use a different approach so it doesn't matter. I'm not sure how our method solved this in the end. As mentioned, the approach is to use Homebrew, in particular, something called pkgconfig. I used a Ruby script that downloads and installs Homebrew, as described here. The install gave this warning:


Warning: The following *evil* dylibs exist in /usr/local/lib
They may break builds or worse. You should consider deleting them:
/usr/local/lib/libfreetype.6.dylib
/usr/local/lib/libpng.3.dylib
/usr/local/lib/libpng12.0.dylib
/usr/local/lib/libz.1.2.5.dylib


These are, of course, the libraries we just installed in order to get matplotlib to build.

That's what got me working on the other method more seriously. I don't think the danger is all that great, but clearly it is better to use libraries that are (i) still available from the maintainers and (ii) have been vetted at least to some extent by other folks including Apple. AFAIK there aren't any known security issues with the versions we installed previously. I ripped them out anyway (they seemed to confuse pkgconfig).

If we ask Homebrew for the libraries, it just points us to the ones we saw in /usr/X11:


> brew search libpng
Apple distributes libpng with OS X, you can find it in /usr/X11/lib.
However not all build scripts look here, so you may need to call ENV.x11
in your formula's install function.

> brew search freetype
Apple distributes freetype with OS X, you can find it in /usr/X11/lib.
However not all build scripts look here, so you may need to call ENV.x11
in your formula's install function.


Following the instructions in the blog post I first installed pkgconfig:


> sudo brew install pkgconfig
Cowardly refusing to `sudo brew'
> brew install pkgconfig

!!

That's all the Homebrew we need. This is followed by:


cd ~/Desktop
git clone git://github.com/matplotlib/matplotlib.git
cd matplotlib/
python setup.py build
sudo python setup.py install


And we can see that I really did overwrite the first matplotlib install, and that we're actually using the libraries from /usr/X11, by first doing


export DYLD_PRINT_LIBRARIES=1


Then when we run a script that imports matplotlib.pyplot, the Terminal shows (among much else) this:


dyld: loaded: /usr/X11/lib/libfreetype.6.dylib
..
dyld: loaded: /usr/X11/lib/libpng15.15.dylib


So that's what I'd recommend and it seems to be working fine. This simple script works exactly as you'd expect.


import matplotlib.pyplot as plt
Y = [1,4,9,16]
plt.scatter(range(len(Y)),Y,s=250,color='r')
plt.savefig('example.png')


So the next thing to do is to figure out pkgconfig and Homebrew work their magic!

P.S. You will still need to make and edit ~/.matplotlibas discussed last time.

12 comments:

Paul Agapow said...

Good stuff. Installing and reinstalling matplotlib (and mysqldb, etc.) has been one of my bugbear task s for years. There's always something new to screw up or fall over. For reference, I've collected some notes here. Using a package manager like homebrew or port et al. to install the dependencies is a godsend.

telliott99 said...

Thanks for reading, and for the link. Note that what we got from Homebrew was pkgconfig. How that helped matplotlib to find the libraries provided by Lion in X11 is still a mystery to be solved.

FĂ©lix-Antoine Fortin said...

Excellent stuff indeed, you saved a lot of time.

I might be able to shed light on the pkgconfig mystery. When installing pkgconfig with Homebrew, the formula add to the pkg-config search path, the ones containing X11 pkg-config files. These files indicate where differente dependencies like libpng and freetype can be found. Since the Python setup use pkg-config when present to solve the dependencies, by installing it with the right path, everything falls into place.

If you are curious, have a look at the pkg-config formula in /usr/local/Library/Formula/pkg-config.rb

Thanks again.

Kosher said...

Hey, this is a great tip! This actually helped with the sam errors in compiling DirectFB on MacOSX Lion. Thank you!

telliott99 said...

Thanks for reading!

bravemouse said...

When I build I get:

unable to execute gcc-4.0: No such file or directory
error: command 'gcc-4.0' failed with exit status 1

I have Xcode installed. Why is this happen?

PLEEASE HELP

telliott99 said...

@bravemouse: something seems to be wrong with your compiler installation. When I do "gcc-4.0 -v" from Terminal, I get "Using built-in specs..." and so on. What do you get?

Charles Feduke said...

Victory! Very helpful, was afraid I'd have to learn everything about Python on OSX to get manbearpig... I mean matplotlib installed. Thanks!

k-genomics said...

Hi telliot

I've got same problem as bravemouse. Mine is gcc-4.2 rather 4.0. How can I solve this problem?

telliott99 said...

@k-genomics: You haven't given me anything to go on. What happens when you paste this into the Terminal:

cat > hello.c << EOF
#include

int main() {
printf("Hello, world!\n");
return 0;
}
EOF

gcc hello.c
./a.out

telliott99 said...

Sorry one line was interpreted as having an html tag:
#include<stdio.h>

Sho Goh said...

I have the same problem as k-genomics.
I typed in terminal as you had instructed and got "Hello, world!"
My gcc is ver 4.3.4.