Monday, November 15, 2010

Installing SciPy with MacPorts


To continue with the previous post (here), I thought it would make life easier if I could install SciPy or other software using MacPorts. Some years have passed since the unpleasantness with Fink (so long that I'm a little hazy now on the details). Anyway, as explained previously, after installing MacPorts, I ran into a problem with not being able to build one of SciPy's dependencies: LAPACK.

MacPorts uses a file called a Portfile to control all of what happens when a package is downloaded and built. The problem stems from the release of a new version of LAPACK. Apparently they didn't tell MacPorts about it, so the Portfile for the atlas port (which includes LAPACK), is out of date. In particular, the Portfile directions allow verification of three checksums on the download (md5, sha-1 and rmd160). Since the Portfile was out of date these didn't match. You can do this manually on any file of interest like so:

md5 filename.tar.gz
openssl sha1 filename.tar.gz
openssl rmd160 filename.tar.gz

The way to fix this issue is to change the Portfile. For example, the MacPorts stuff is mostly or all in

$ ls /opt/local/var/macports
build logs port-help.tcl registry sources
distfiles packages receipts software

and the Portfile is there:

ls /opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas
Portfile Portfile.orig Portfile.rej files

A standard approach would be to verify that the download obtained by MacPorts is authentic, by verifying the checksums against a download you obtain yourself, or values published by a trusted source, then edit the MacPorts file to change the values we're checking against. One could simply copy this modified Portfile and replace the original. However, the correct method is a little different, allowing one to easily reverse the patch if necessary later.

The "manual" method creates a Portfile Patch by doing a diff between the copy and the original, and then calling patch:


cp -p Portfile Portfile.orig
diff -u Portfile.orig Portfile > Portfile-atlas-fixed.diff
patch -p0 < ~/Desktop/Portfile-atlas-fixed.diff


However, then I ran into a second problem, which was that the new version of LAPACK was extracted and configured but there was problem later in the process (it wasn't actually in the build phase but (perhaps) in clean). Since the LAPACK release is so new, I guess that nothing will depend on it being version 3.3.0, though a lot has changed. Rather than figure out what the issue is, I edited the Portfile in a different way, keeping the old checksums, and changing the download we request. That's the screenshot at the top of the post. Textmate knows a diff file when it see one, and syntax colors appropriately. That answers the question I posed at the end last time (here).

By the way, the basic reason for the checksum mismatch is that the standard LAPACK download file does not have a version number, at least the tgz doesn't. If we'd asked for the download by version number, we'd have got the correct file. The "stealth upgrade" is because the default download is unnumbered.

Whether I did the direct copy or the patch thing on the version that worked, I don't exactly remember now. patch has an issue that it wants to know whether a previous patch needs to be reversed or not, and can fail. Not sure now, but a direct copy will certainly work.

Now we use a nifty trick involving a shell variable (I think that's what it's called) to cd into where we need to be, and finish what we're trying right now:

$(port dir atlas)
-bash: /opt/local/var/macports/sources/rsync.macports.org/release/ports/math/atlas: is a directory

cd $(port dir atlas)
sudo cp ~/Desktop/Portfile Portfile
c-98-236-78-154:atlas telliott_admin$ sudo port install atlas
Portfile changed since last build; discarding previous state.
---> Computing dependencies for atlas
---> Fetching atlas
---> Verifying checksum(s) for atlas
---> Extracting atlas
---> Applying patches to atlas
---> Configuring atlas
---> Building atlas
---> Staging atlas into destroot
---> Installing atlas @3.8.3_4+gcc44
---> Activating atlas @3.8.3_4+gcc44
---> Cleaning atlas
c-98-236-78-154:atlas telliott_admin$

Yes!

sudo port install py26-scipy @0.8.0
..
---> Fetching py26-scipy
---> Attempting to fetch scipy-0.8.0.tar.gz from http://superb-east.dl.sourceforge.net/scipy
---> Attempting to fetch scipy-0.8.0.tar.gz from http://downloads.sourceforge.net/scipy
---> Verifying checksum(s) for py26-scipy
---> Extracting py26-scipy
---> Configuring py26-scipy
---> Building py26-scipy
---> Staging py26-scipy into destroot
---> Installing py26-scipy @0.8.0_0+gcc44
---> Activating py26-scipy @0.8.0_0+gcc44
---> Cleaning py26-scipy

$ /opt/local/bin/python2.6
Python 2.6.6 (r266:84292, Nov 14 2010, 17:37:27)
[GCC 4.2.1 (Apple Inc. build 5664)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from scipy import *
>>>

How cool is that?