Playing with SciPy (install part 3)

If you've been reading my posts (one, two, three) looking for help on intalling SciPy, I have two things to say:

• it seems to be working on my MacBook Pro
• I think it shouldn't be

If your goal is to actually get it working on your system, I suggest you go back to the instructions from hyperjeff here.

Let's review where we were:

• I built Python 2.6.4 from source. I was having trouble with specifying the correct flags to do a framework build, so I skipped that step (at first). This Python is in

$ /usr/local/bin/python2.6 Python 2.6.4 (r264:75706, Jan 31 2010, 16:39:21) $ file /usr/local/bin/python2.6 /usr/local/bin/python2.6: Mach-O 64-bit executable x86_64

A sharp-eyed reader spotted the problem with the flags I was passing to ./configure (a silly typo), so that I was able to make a "framework" build.

I'm not actually sure where this Python ended up. There is a second installed Python (a "framework" version) in

$ /Library/Frameworks/Python.framework/Versions/2.6/bin/python2.6 Python 2.6.4 (r264:75821M, Oct 27 2009, 19:48:32) [GCC 4.0.1 (Apple Inc. build 5493)] on darwin

but it has the wrong date (and gcc version) to be the one I built.

gcc --version i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5646) (dot 1)

Ned Deily cautions that there are important components that need to be added after building Python. I'll have to look into that and report back.

And of course, I still have the Apple-installed "system" Python

$ python Python 2.6.1 (r261:67515, Jul 7 2009, 23:51:51)

I put a sym link into ~/bin pointing to the new (non-framework) Python

$ ls ~/bin/python26 /Users/telliott_admin/bin/python26 $ python26 Python 2.6.4 (r264:75706, Jan 31 2010, 16:39:21)

---

In addition to the Python install saga:

• I installed a fortran compiler (notes here).

$ gfortran --version GNU Fortran (GCC) 4.2.3

There are pieces of Numpy (and perhaps SciPy and perhaps other things) written in fortran. Unfortunately the more classic f77 compiler and newer gfortran compilers do not play nice with each other, so we have to build everything with the same compiler.

---

• I installed FFTW (notes here)
Check out the documentation, which is extensive and very well-done

• I worked through notes for installing UMFPACK and its friends (AMD and UFconfig) here. I was unclear whether this step is required. I skipped it on my MacBook, and I think it worked (although there is some uncertainty about what exactly happened, as you will see). On my iMac at work building Numpy failed complaining about no UMFPACK. I'm going to leave that as an unsolved mystery for now. In particular, it's not clear to me why there's no make install in the instructions, but perhaps it's taken care of by the unusual make hb.

• Following hyperjeff, I built Numpy after setting some environment variables.

export MACOSX_DEPLOYMENT_TARGET=10.6 export CFLAGS="-arch i386 -arch x86_64" export FFLAGS="-m32 -m64" export LDFLAGS="-Wall -undefined dynamic_lookup -bundle -arch i386 -arch x86_64"

Notably, I did not do his fifth one (because I'm not using the Framework Python).

export PYTHONPATH="/Library/Python/2.6/site-packages/"

svn co http://svn.scipy.org/svn/numpy/trunk numpy cd numpy python26 setup.py build --fcompiler=gnu95 sudo python26 setup.py install

Here is where I may have screwed up. I forgot to modify site.cfg (see hyperjeff's instructions), before doing the build step above. But initial tests seemed to work.

• Following hyperjeff, I built SciPy from the subversion checkout. And installed it.

---

So, here's the wild thing. I have numpy in my new Python:

$ python26 Python 2.6.4 (r264:75706, Jan 31 2010, 16:39:21) [GCC 4.2.1 (Apple Inc. build 5646) (dot 1)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.__version__ '1.5.0.dev8081' >>> numpy.__file__ '/usr/local/lib/python2.6/site-packages/numpy/__init__.pyc' >>> import scipy Traceback (most recent call last): File "", line 1, in ImportError: No module named scipy

But not SciPy. Even though I built it with python26, it is not in the correct site-packages directory. It is in the system one!

$ python Python 2.6.1 (r261:67515, Jul 7 2009, 23:51:51) [GCC 4.2.1 (Apple Inc. build 5646)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import numpy >>> numpy.__version__ '1.3.0' >>> numpy.__file__ '/Library/Python/2.6/site-packages/numpy-1.3.0-py2.6-macosx-10.6-universal.egg/numpy/__init__.pyc' >>> import scipy >>> scipy.__version__ '0.7.1' >>> scipy.__file__ '/Library/Python/2.6/site-packages/scipy/__init__.pyc'

drwxr-xr-x 39 root admin 1326 Jan 31 17:53 scipy -rw-r--r-- 1 root admin 1715 Jan 31 17:53 scipy-0.7.1-py2.6.egg-info

>>> from scipy import * >>> A = randn(100) >>> A.shape = (10,10) >>> mean(A, axis=1) array([ 0.16384552, 0.37521045, -0.74245801, -0.07997456, 0.69394119, -0.57582111, -0.0668192 , 0.06084532, -0.0946143 , 0.24184168])

How the heck did that happen? Is it possible that all these contortions were unnecessary? Here's the kicker: we I may never know. Because I'm ready to explore SciPy!

[UPDATE: Quick test.
In the code using a two sample t-test from the other day, substitute

import scipy.stats.stats as stats t, prob = stats.ttest_ind(nums1, nums2)

Output:

/Library/Python/2.6/site-packages/scipy/stats/stats.py:420: DeprecationWarning: scipy.stats.mean is deprecated; please update your code to use numpy.mean. Please note that: - numpy.mean axis argument defaults to None, not 0 - numpy.mean has a ddof argument to replace bias in a more general manner. scipy.stats.mean(a, bias=True) can be replaced by numpy.mean(x, axis=0, ddof=1). axis=0, ddof=1).""", DeprecationWarning) avg 0.0500, std 0.0090 min 0.0300, max 0.0680

Huh. Looks like it works. Glad I didn't pay for it.