Monday, January 31, 2011
Jake Shimabukuro
wikimedia
I guess I'm probably the last person in the world to learn about this guy:
one
two
three
four
five
my favorite:
six
Incredible feeling...
one more
and the last
but one.
(Can't help myself)
Unix spoken here (4)
There are a just few more Unix commands I should mention before we exhaust everyone's patience. (Three posts here, here, and here).
I'm sure you're familiar with
Don't forget to read the
Many Unix commands appear simple but have numerous options and arguments that make them more like a Swiss Army Knife than a pocketknife. Output on the command line using
For some reason you need to format the output from
Nearly the whole alphabet! But doing this without the
Try
There is also
Or drop
There's a command to find out information about different processes with finer grained control than Activity Viewer. For example:
Say:
The
I "find" all the files under my home directory and feed the results to
Sometimes I just want to use Python as a calculator or get an ascii value, but without calling up the interpreter (and having to dismiss it):
Finally, here are a few more that either I've found useful or I'm sure they will be, but I don't feel expert enough to try to explain them. Have fun exploring:
If you get bored here; or try to check out how Greg's email address gets generated!
And there's always
I'm sure you're familiar with
sudo
, which stands for superuser, do once. If not, you can see wikipedia. Normally you will use this to write (install) or even delete files in /usr/bin
and the like. > mv /usr/bin/znew ~/Desktop/znew |
Don't forget to read the
man
page on this one. There is always something interesting in the manual.Many Unix commands appear simple but have numerous options and arguments that make them more like a Swiss Army Knife than a pocketknife. Output on the command line using
man
can be broken up into chunks or streamed using the down arrow, but maybe you want to study it or keep it as a reference. For some reason you need to format the output from
man
in a special way before directing it to a file, for example:man ls | col -bx > results.txt |
Nearly the whole alphabet! But doing this without the
col
leads to selective stuttering:LS(1) BSD General Commands Manual LS(1) |
head
and tail
are useful for looking at the first few lines or last few lines of a file. less
is like what you get when you just do man some_command
---it gives chunks or streams.Try
man cat
to see what else that can do.There is also
split
but I couldn't figure out how to make it take my favorite separator ('\n\n').diff
does what you might expect. The use case might be that you have some good edits but also some bad edits to a file, so you want to revert to the old one, and then try the edits one by one. > diff old.txt new.txt > results.txt |
Or drop
results.txt
onto TextMate for a little color (graphic).There's a command to find out information about different processes with finer grained control than Activity Viewer. For example:
ps -u username |
Say:
> ps -p 48068 -O %mem |
The
man
page on ps
is extensive.find
is tricky. I need a lot more practice to understand it. Here's a simple one, and then a fun one though:> find . -name "old.txt" -print |
I "find" all the files under my home directory and feed the results to
wc
(word count), discovering that there are 118,791 lines in the output---that's the number of files! This link seems to have good info on find
.Sometimes I just want to use Python as a calculator or get an ascii value, but without calling up the interpreter (and having to dismiss it):
> python -c 'print 3.0/17' |
> hexdump -C post.txt |
Finally, here are a few more that either I've found useful or I'm sure they will be, but I don't feel expert enough to try to explain them. Have fun exploring:
alias |
If you get bored here; or try to check out how Greg's email address gets generated!
And there's always
ssh
. But that really does deserve a post of its own.
Unix spoken here (3)
Let's do some more with Unix commands. Previous posts here and here.
You might have noticed that I have a very short prompt in Terminal, just the character `>` followed by a space. That's because I have an (invisible) file in my home directory named
We can change back to the original behavior by changing the name of the file or moving it, and then re-starting Terminal:
We could comment out that line of the file (now that it's visible with no leading `.` in the filename, we can easily open it in TextEdit).
But I like it, so I'll revert to the original situation.
I think there is a way to get TextEdit to show hidden files in the Open dialog, but I forget how at the moment. In any event, it's interesting that having opened and closed the file while it was visible on the Desktop, Open Recent now shows
Another way to edit the file is to use
I'm sure you can figure out how to use
One more useful thing in
When I call
If I wanted to modify the $PATH variable I could do:
My $PATH is pretty complicated for several reasons: use of MacPorts, some things related to ncbi and X11, etc. It would take too long to explain.
The shell has command history. If you want to repeat a command, just press the up arrow key. To repeat a command further back, keep pushing until you see the one you want. Then press Return. Suppose you have a series of commands (say 3 in a row), and the first of these is 6 commands ago. Press the arrow 6 times, then return. Press the arrow 6 times, then return. Press the arrow 6 times, then return. Because the command history is updated each time, 6 arrow presses gets you back exactly where you need to be for each of the three commands in the sequence.
Alternatively you could do
which lists each command I've entered since the last time I booted my machine (even saves between Terminal sessions). One can select a command from the history by number or by partial name match:
First it prints the command, and then the result. You can combine commands on a single line with `;`
We're getting pretty good with this stuff! We don't really want to list all 500+ commands in the history, just the last two. So we feed the results of
The Unix command
Here, I edited some file with pico but I can't remember which one it was. So the entire output of the
I won't show you the file.
The last command for this post is
Now the listing for foo shows an `l` as the first character in the file mode, and it also shows
You might have noticed that I have a very short prompt in Terminal, just the character `>` followed by a space. That's because I have an (invisible) file in my home directory named
.bash_profile
, containing this line:> cat ~/.bash_profile |
We can change back to the original behavior by changing the name of the file or moving it, and then re-starting Terminal:
mv ~/.bash_profile ~/Desktop/x.txt |
c-98-236-78-154:Desktop telliott$ |
We could comment out that line of the file (now that it's visible with no leading `.` in the filename, we can easily open it in TextEdit).
#PS1="> "; export PS1 |
But I like it, so I'll revert to the original situation.
I think there is a way to get TextEdit to show hidden files in the Open dialog, but I forget how at the moment. In any event, it's interesting that having opened and closed the file while it was visible on the Desktop, Open Recent now shows
.bash_profile
as an option.Another way to edit the file is to use
pico
(a command line editor):> pico ~/.bash_profile |
I'm sure you can figure out how to use
pico
.One more useful thing in
.bash_profile
is modification to the $PATH variable. Let's look at that from the command line:> echo $PATH |
When I call
ls
, for example, these directories are searched in turn until the program is discovered finally. It is in:> whereis ls |
If I wanted to modify the $PATH variable I could do:
> PATH=~/Desktop:$PATH; export PATH |
My $PATH is pretty complicated for several reasons: use of MacPorts, some things related to ncbi and X11, etc. It would take too long to explain.
The shell has command history. If you want to repeat a command, just press the up arrow key. To repeat a command further back, keep pushing until you see the one you want. Then press Return. Suppose you have a series of commands (say 3 in a row), and the first of these is 6 commands ago. Press the arrow 6 times, then return. Press the arrow 6 times, then return. Press the arrow 6 times, then return. Because the command history is updated each time, 6 arrow presses gets you back exactly where you need to be for each of the three commands in the sequence.
Alternatively you could do
history
:> history |
which lists each command I've entered since the last time I booted my machine (even saves between Terminal sessions). One can select a command from the history by number or by partial name match:
> !505 |
First it prints the command, and then the result. You can combine commands on a single line with `;`
> !504; !512 |
We're getting pretty good with this stuff! We don't really want to list all 500+ commands in the history, just the last two. So we feed the results of
history
to the Unix tool called tail
through a `|`, a Unix "pipe." tail
takes an option for how many lines at the end of the "file" to display.The Unix command
grep
searches lines of text for those matching a pattern. It's particularly useful in combination with `|`.> history | grep pico |
Here, I edited some file with pico but I can't remember which one it was. So the entire output of the
history
is fed through the pipe to grep
, searching for lines containing the pattern. Now I can just do:> !40 |
I won't show you the file.
The last command for this post is
ln
with the option -s
which makes a "soft link" to a file. I can never remember, but you want to put the existing file first (that's the opposite of the way I would design it):> cat temp/y.txt |
Now the listing for foo shows an `l` as the first character in the file mode, and it also shows
-> temp/y.txt
, where it links to.
Sunday, January 30, 2011
Unix spoken here (2)
Let's continue with an introduction to Unix commands. First post here. And here are two more good links (one, two).
Let's start by putting some data into a file. We can use the command
Suppose we wish to move this file to a new sub-directory. First we construct the new directory and then we move the file with
Let's make the file `x.txt` actually do something, e.g.:
Note that we wrote over the previous contents of
The
We can also use the she-bang thing (#!) to make the code in a file executable. Use TextEdit to save the following code in a file y.txt on the Desktop:
Then we do the following. An executable can be executed just by entering its path, but.. there is a restriction that you can't do that for a program in the current directory. You must specify it as the relative path from the directory where we are currently
What has happened? We need to make the file "executable" by setting the "permission bits." Without getting into too much detail, these are output using a new option with the
The three permissions are r (read), w (write) and x (execute), listed in order for the user, her group, and the world. 7 is all of them. `x` is 1, `w` is 2, `r` is 4, and the combinations result from addition.
There are various shorthands for this. First reverting the previous change, then we could do this:
We have given the user (only) permission to execute the file.
When we do
From
And finally, ls also has a recursive option, which can be grouped into
Let's continue by copying this file into our
On the other hand,
Once again, note the lack of warnings. The old copies of
Eventually, we tire of this. First we try to delete a single file:
But maybe
The `-r` option is really dangerous. The classic Unix "joke" is to tell someone to do some combination of
You might think about looking at the
Let's start by putting some data into a file. We can use the command
echo
and the `>` redirection operator. Take a look at what's there with cat
:> cd |
Suppose we wish to move this file to a new sub-directory. First we construct the new directory and then we move the file with
mv
:> mkdir temp |
Let's make the file `x.txt` actually do something, e.g.:
> echo "echo xyz" > temp/x.txt |
Note that we wrote over the previous contents of
temp/x.txt
and the file system did not warn us first. There is no way to recover from this if it was a mistake!The
sh
command executes the text in the file you give it; in this case the output is familiar. Normally, we'd probably use the file extension .sh
for such a script, but it's not required. (Even with Python scripts, the .py
extension is only required if you want to import a module).We can also use the she-bang thing (#!) to make the code in a file executable. Use TextEdit to save the following code in a file y.txt on the Desktop:
#! sh |
Then we do the following. An executable can be executed just by entering its path, but.. there is a restriction that you can't do that for a program in the current directory. You must specify it as the relative path from the directory where we are currently
./executable
where the `.` means this directory (the restriction has never made a lot of sense to me either, but it's apparently a security hole):> cat y.txt |
What has happened? We need to make the file "executable" by setting the "permission bits." Without getting into too much detail, these are output using a new option with the
ls
command:> ls -l y.txt |
The three permissions are r (read), w (write) and x (execute), listed in order for the user, her group, and the world. 7 is all of them. `x` is 1, `w` is 2, `r` is 4, and the combinations result from addition.
There are various shorthands for this. First reverting the previous change, then we could do this:
> chmod 644 y.txt |
We have given the user (only) permission to execute the file.
> ls -l ~/Desktop |
When we do
ls
on the Desktop directory (above), we get a listing for both the temp
sub-directory and the file. Note that the first character in the "file mode" is d
for the directory. staff
is my group name, and after that comes the number of bytes in each "file", including directories.From
man ls
The Long Format
If the -l option is given, the following informa-
tion is displayed for each file: file mode, num-
ber of links, owner name, group name, number of
bytes in the file, abbreviated month, day-of-
month file was last modified, hour file last mod-
ified, minute file last modified, and the path-
name. In addition, for each directory whose con-
tents are displayed, the total number of 512-byte
blocks used by the files in the directory is dis-
played on a line by itself, immediately before
the information for the files in the directory.
If the file or directory has extended attributes,
the permissions field printed by the -l option is
followed by a '@' character. Otherwise, if the
file or directory has extended security informa-
tion (such as an access control list), the per-
missions field printed by the -l option is fol-
lowed by a '+' character.
And finally, ls also has a recursive option, which can be grouped into
-lR
:> ls -lR |
Let's continue by copying this file into our
temp
directory:> cp y.txt temp |
On the other hand,
mv
actually "moves" the file:> mv y.txt temp |
Once again, note the lack of warnings. The old copies of
y.txt
on the Desktop (and in temp
) are gone forever.Eventually, we tire of this. First we try to delete a single file:
> rm temp/y.txt |
But maybe
temp
has lots of files, so we try:> rmdir temp |
The `-r` option is really dangerous. The classic Unix "joke" is to tell someone to do some combination of
rm
and -r
and *
. Don't fall for it.You might think about looking at the
man
page for rm
and maybe using the -i
option.
Unix spoken here (1)
There are probably hundreds of pages on the web with good info about basic Unix commands (e.g. here, here). I thought I'd give some of the commands I think are most useful, but I have to start at the beginning. So, here goes with Unix in 5 minutes, Unleashed, For Dummies..
The first command,
The "shell" is smart and has "command-completion." For example, from my home directory I enter
Another very common command is
Any directory may have "hidden" files whose names are prefaced by a dot `.`. These can be shown with
The shell understands paths to files in both an absolute and relative sense. A path starting with
The character
Sometimes we want to make a jump in the file system but intend to come right back.
As one more example of using options or arguments to a command,
Finally, to move toward the tips of the file system tree, you specify the directory at the next level so it has to be explicit, but there is only one parent, which we can reach implicitly with
> cd |
The first command,
cd
, says to change directory; with no other argument given, we go to our home directory. To see where we are in the file system we can do pwd
: print working directory. Also, the ~ symbol is a shorthand for the user's home directory.The "shell" is smart and has "command-completion." For example, from my home directory I enter
cd De
and then press tab. The shell fills out "Desktop."Another very common command is
ls
: list the contents of the current directory (since there is no argument). And to see the contents of a file you might do cat
with the results shown.> ls -a |
Any directory may have "hidden" files whose names are prefaced by a dot `.`. These can be shown with
ls -a
. There are many (many) other options with results cataloged under man some_command
.The shell understands paths to files in both an absolute and relative sense. A path starting with
/
is absolute, otherwise it's relative. Hence /usr/bin
is a directory starting from the root of the file system, while Desktop
is only understood as a valid destination if I'm in my home directory (or some other directory with Desktop as a sub-directory).The character
*
is a wild card and matches anything; ls /usr/bin/pyth*
lists all the entries matching pyth- + something in the given directory. > pushd /usr/bin |
Sometimes we want to make a jump in the file system but intend to come right back.
pushd
and popd
are useful here, with results as shown.As one more example of using options or arguments to a command,
cat
takes the option -n
, which numbers the output lines. cal
is probably faster than finding the Calendar.app, since I don't usually have it in the Dock.Finally, to move toward the tips of the file system tree, you specify the directory at the next level so it has to be explicit, but there is only one parent, which we can reach implicitly with
cd ..
, meaning "go up."
Saturday, January 29, 2011
Distributed Objects on OS X (2)
I struggled for a while to get Distributed Objects working between different machines (first post here for same-machine example). According to the docs and this, you should just use
But it didn't work for me. So I asked a question on SO (here), and asked the Google. The latter approach finally worked. I got a working example with code from here. The discussion in the docs for the NSConnection class helped too (see
[UPDATE: On reflection, it does seem a bit weird, because a Mach port is supposed to be a one-way channel, yet these are clearly two way. I guess they are probably complex objects built from multiple NSMacPorts. ]
Server:
Note: if you're not just killing the Server (as I did) you should do invalidate on the connection (and I guess the ports too?).
Client
Output:
It works!
NSSocketPortNameServer *name_server; |
But it didn't work for me. So I asked a question on SO (here), and asked the Google. The latter approach finally worked. I got a working example with code from here. The discussion in the docs for the NSConnection class helped too (see
initWithReceivePort:sendPort:
. If you set it up with nil for the sendPort, it uses the same port for both sending and receiving, which is what you want.)[UPDATE: On reflection, it does seem a bit weird, because a Mach port is supposed to be a one-way channel, yet these are clearly two way. I guess they are probably complex objects built from multiple NSMacPorts. ]
Server:
// gcc talk3.m -o test -framework Foundation |
Note: if you're not just killing the Server (as I did) you should do invalidate on the connection (and I guess the ports too?).
Client
listen.py
is on my laptop:from Foundation import * |
Output:
> python listen.py |
It works!
NSTask again
I have a bit more to explore about NSTask (previous post here). There is some discussion on SO about the difference between an NSTask and an NSThread (here).
The script
The first attempt just launches the task without setting it up first.
The second example does the setup. The path p2 (to Python directly) isn't used in these versions but I left it in. For a while, I was getting errors when I tried to setup the task and then launch it unless I fed it to Python first and gave the script path as an argument. Then, the errors just stopped. I can't recreate the problem. This kind of bug that "just disappeared" happened with the environment dictionary as well. No clue as to why.
Of note here is that the arguments must be in a list or array, and that the int argment must be converted to a string. The error if not is weird (but at least we're pointed to the correct line):
Otherwise we get this:
Next, we instantiate the task and then set its arguments and environment, having modified the dict to have an extra key/value pair:
The ultimate line of output is from the task and shows that we did get the modified environment.
And in the last example, we use an NSFileHandle and redirect standard output. This could be useful in the case where you don't have full control over the task that is being executed. I found that
The redirect is good, but I couldn't get the file read to work. I tried before and after launch, and I tried re-grabbing the file handle, and I also tried
[UPDATE: Perhaps the answer has something to do with NSPipe (here). ]
The script
task.py
uses the #! thing and is set to be executable (by the user only). The code looks for an int in argv and a second arg that tells whether to examine the environment variable dictionary.#! /usr/bin/python |
The first attempt just launches the task without setting it up first.
The second example does the setup. The path p2 (to Python directly) isn't used in these versions but I left it in. For a while, I was getting errors when I tried to setup the task and then launch it unless I fed it to Python first and gave the script path as an argument. Then, the errors just stopped. I can't recreate the problem. This kind of bug that "just disappeared" happened with the environment dictionary as well. No clue as to why.
from Foundation import * |
Of note here is that the arguments must be in a list or array, and that the int argment must be converted to a string. The error if not is weird (but at least we're pointed to the correct line):
2011-01-29 03:18:00.109 Python[19298:60f] -[OC_PythonNumber fileSystemRepresentation]: unrecognized selector sent to instance 0x101a6d950 |
Otherwise we get this:
> python parent.py |
Next, we instantiate the task and then set its arguments and environment, having modified the dict to have an extra key/value pair:
D = NSProcessInfo.processInfo().environment() |
> python parent.py |
The ultimate line of output is from the task and shows that we did get the modified environment.
And in the last example, we use an NSFileHandle and redirect standard output. This could be useful in the case where you don't have full control over the task that is being executed. I found that
NSFileHandle.fileHandleForWritingAtPath_
apparently requires the file to already exist.p3 = NSHomeDirectory() + '/Desktop/results.txt' |
> python parent.py |
The redirect is good, but I couldn't get the file read to work. I tried before and after launch, and I tried re-grabbing the file handle, and I also tried
readDataToEndOfFile
. Not sure what's up with that yet.[UPDATE: Perhaps the answer has something to do with NSPipe (here). ]
Thursday, January 27, 2011
Decorators in Python
There is a nice simple post about Python decorators from Bruce Eckels (here; discussion on Stack Overflow here, docs here, more here). I copied some of his example code and show it below, but first, the output of the run:
The main point is that a decorator is a kind of macro for functions. It changes them, and can do so in almost unlimited ways. The output above shows that the decorator code can be executed before or after the function itself is called. (And he notes that you don't even have to do that!) They're used quite a bit in PyObjC (e.g.
As Bruce says:
> python script.py |
The main point is that a decorator is a kind of macro for functions. It changes them, and can do so in almost unlimited ways. The output above shows that the decorator code can be executed before or after the function itself is called. (And he notes that you don't even have to do that!) They're used quite a bit in PyObjC (e.g.
@objc.signature('v@0:@8')
). I should probably explore all of that in another post.As Bruce says:
Decorators allow you to inject or modify code in functions or classes
The only constraint upon the object returned by the decorator is that it can be used as a function -- which basically means it must be callable. Thus, any classes we use as decorators must implement __call__.
What should the decorator do? Well, it can do anything but usually you expect the original function code to be used at some point. This is not required, however.
class entryExit(object): |
A peek into the Objective-C "runtime"
I'm playing with Cocoa today. I ran across a post (here), reminding me that all our fancy "messages" get changed by the Objective-C "runtime." The real call is
Note: you have to use an int variable, you can't just pass an int directly to
Output:
objc_msgSend
and you can do that yourself if you want:// gcc test.m -o test -framework Foundation |
Note: you have to use an int variable, you can't just pass an int directly to
objc_msgSend
.Output:
> ./test |
Distributed Objects on OS X
I got a simple example with Distributed Objects working. This is supposed to be the standard way to communicate between processes. I tried it first with both the Server and the Client in PyObjC. Eventually I rewrote the Server in Objective-C and then it worked.
[UPDATE: Found a great working example (old, but from an Apple engineer here). ]
The output is:
I'm not sure what the problem was but the error I got was
Server:
Run with:
You'll have to kill the process when you're done. If you have tried once already in Terminal, you will need to Quit Terminal and Re-launch. Otherwise
Client:
I also tried to connect to another machine on my local network, but I couldn't get that to work either. The
[UPDATE: Found a great working example (old, but from an Apple engineer here). ]
The output is:
> python listen.py |
I'm not sure what the problem was but the error I got was
[OC_PythonString initWithBytes:length:encoding:]: unrecognized selector sent to instance 0x3635130 |
Server:
// gcc talk.m -o test -framework Foundation |
Run with:
> ./test |
You'll have to kill the process when you're done. If you have tried once already in Terminal, you will need to Quit Terminal and Re-launch. Otherwise
registerName:
will fail.Client:
from Foundation import * |
I also tried to connect to another machine on my local network, but I couldn't get that to work either. The
proxy_obj
was nil/None.
Wednesday, January 26, 2011
NSNotifications
I want to send information between two running applications and I mistakenly thought I could use an NSNotification to do this. [See the UPDATE].
So this post is about the first step, getting an NSNotification to work within a running app. The following is from an Xcode Cocoa-Python app, and in AppDelegate.py we have:
There's a button in the window hooked up to the IBAction. I push the button and get:
But I couldn't get it to work between apps, for example with this one compiled in Terminal:
I asked on Stack Overflow (here), and they say I should be using Distributed Objects (here). So I have to look into that.
[ UPDATE: Just substitute NSDistributedNotificationCenter in the code above. It works! ]
So this post is about the first step, getting an NSNotification to work within a running app. The following is from an Xcode Cocoa-Python app, and in AppDelegate.py we have:
from Foundation import * |
There's a button in the window hooked up to the IBAction. I push the button and get:
<NSButton: 0x61c7b0> button |
But I couldn't get it to work between apps, for example with this one compiled in Terminal:
// gcc tell.m -o test -framework Foundation |
I asked on Stack Overflow (here), and they say I should be using Distributed Objects (here). So I have to look into that.
[ UPDATE: Just substitute NSDistributedNotificationCenter in the code above. It works! ]
Saturday, January 22, 2011
Nullspace of a vector
This post is a note to myself about linear algebra.
Let's consider two vectors in R2:
We can see from a plot that these vectors a and b are obviously not pointing in the same direction.
Also we notice that a is not a multiple (or a linear combination) of b (or vice versa). There is no constant such that when the elements of a are multiplied by the constant we get b.
Now consider a point P in R2, say
We can construct a linear combination of a and b that reaches this P or any other point.
How? Simply place one of the vectors at the origin and move along it (perhaps in a negative direction), and place the second vector at P and move along it, and find where the two lines meet.
Call the coordinates of the point where the lines cross x and y. There are actually two possibilities depending on which vector we choose for each role. Suppose we move in the reverse direction from the origin along a and from P along b. From the point-slope equation we know that:
We can solve these two equations pretty easily. From the first
and from the second:
The other solution is symmetrical (we're dealing with a parallelogram). We would need to go +2 across and +8/3 up from P to reach x, y = (4, -4/3).
Suppose we want to know the actual multipliers for a and b, i.e. the fraction of the length of a and b that we travel along each vector.
From the figure, we can estimate that the values will be about -0.7 and -1.25.
Call these multipliers u and v. In matrix language
One way we can solve the system (call it the Algebra 2 way) is as follows:
Solve the second equation for v:
Plug into the first equation:
Finally, consider P not just as a point but as a vector c:
Since we could have chosen P to be any point, c could be any vector in the x,y-plane and it would be constructed as a linear combination of a and b:
We say that c is in the column space of the matrix whose columns are the vectors a and b:
because we can get to c by taking a linear combination of the columns of M.
And of course, having a + b = c, we can also add -c to get:
If we think of another matrix as the column vectors a, b and c lined up:
The combination u = -2/3, v = -4/3, w = -1 solves this equation and brings us back to zero:
That solution (a vector in R3):
is said to be in the nullspace of
because Ax = 0.
Let's consider two vectors in R2:
a = ( 3, 4) = [ 3 ] |
We can see from a plot that these vectors a and b are obviously not pointing in the same direction.
Also we notice that a is not a multiple (or a linear combination) of b (or vice versa). There is no constant such that when the elements of a are multiplied by the constant we get b.
Now consider a point P in R2, say
P = (2,-4) |
We can construct a linear combination of a and b that reaches this P or any other point.
How? Simply place one of the vectors at the origin and move along it (perhaps in a negative direction), and place the second vector at P and move along it, and find where the two lines meet.
Call the coordinates of the point where the lines cross x and y. There are actually two possibilities depending on which vector we choose for each role. Suppose we move in the reverse direction from the origin along a and from P along b. From the point-slope equation we know that:
(0 - y) / ( 0 - x) = 4/3 (from a) |
We can solve these two equations pretty easily. From the first
y = 4/3 x |
and from the second:
(-12 - 3y) = -(2 - x) |
The other solution is symmetrical (we're dealing with a parallelogram). We would need to go +2 across and +8/3 up from P to reach x, y = (4, -4/3).
Suppose we want to know the actual multipliers for a and b, i.e. the fraction of the length of a and b that we travel along each vector.
From the figure, we can estimate that the values will be about -0.7 and -1.25.
Call these multipliers u and v. In matrix language
M = [ 3 -3 ] |
One way we can solve the system (call it the Algebra 2 way) is as follows:
3u - 3v = 2 |
Solve the second equation for v:
v = -4u - 4 |
Plug into the first equation:
3u - 3(-4u - 4) = 2 |
Finally, consider P not just as a point but as a vector c:
a = ( 3, 4) |
Since we could have chosen P to be any point, c could be any vector in the x,y-plane and it would be constructed as a linear combination of a and b:
u a + v b |
We say that c is in the column space of the matrix whose columns are the vectors a and b:
M = [ 3 -3 ] |
because we can get to c by taking a linear combination of the columns of M.
And of course, having a + b = c, we can also add -c to get:
u a + v b - c = 0 |
If we think of another matrix as the column vectors a, b and c lined up:
[ 3 -3 2 ] [ u ] = [ 0 ] |
The combination u = -2/3, v = -4/3, w = -1 solves this equation and brings us back to zero:
-2/3 [ 3 ] -4/3 [ -3 ] - [ 2 ] = [ 0 ] |
That solution (a vector in R3):
x = [ u ] = [ -2/3 ] |
is said to be in the nullspace of
A = [ 3 -3 2 ] |
because Ax = 0.
Thursday, January 20, 2011
Sal Khan rocks
I'm always the last person to learn about cool new stuff, so when I say I was scouting around on the web today and ran into Sal Khan and his Khanacademy, I expect you will say, I know about him already. I mean, it's 10,000,000 downloads and counting, or something.
If you don't know, the site consists of a very large number of videos on math, science, economics and a few other topics (here). Since Sal's been interviewed a lot, if I still watched PBS reliably, I would've known about him already. But I decided a while ago to watch as little TV as possible.
Several things to say about the material:
(1) the scope is incredible
(2) the depth is also incredible
(3) the quality is excellent
I'm using it to flesh out what I'm getting from Gil Strang on Linear Algebra.
He also seems to be a really nice guy. Check it out!
Wednesday, January 19, 2011
NSCountedSet
I just got a helpful comment on an old post (here) suggesting the use of an
there are several others including:
Let's show off the last one, from PyObjC. Here's the listing:
and the output:
It feels a bit strange, not only can you get the
NSCountedSet
. I'd never heard of this class, but it's described in the Apple docs about Collections (here). Besides the usual charactersNSArray |
there are several others including:
NSPointerArray |
Let's show off the last one, from PyObjC. Here's the listing:
from Foundation import * |
and the output:
x 1 |
It feels a bit strange, not only can you get the
count
of an object that's not in the set (as in Python), but as the docs say:removeObject:
does nothing if anObject is not present in the set
Sunday, January 16, 2011
One space or two
It's a little depressing but also undeniable that I am getting to be an old fart. My birthyear satisfies this equation:
For today what it means is that I learned to type on a typewriter, an archaic device you can read about in wikipedia (here). My first two were unpowered. In typing class, my teacher was very strict: it was an error if you did not have two spaces separating sentences. In typography, or even when typing on a computer, the rules have changed, but legions of us still type to please the critical eye of someone like Brother McDermott.
Apparently that drives certain people crazy (here) or just leaves them bemused (here and here).
I still do it, as you can see it in the source for the blog. It's arguably correct for non-proportional fonts. But, in the end it doesn't matter because html simply ignores the extra space.
What does matter and I find incredibly useful is to separate groups of multi-line data by a double newline. Then I can just do:
It's mildly irritating when I send off a FASTA file in that format and it comes back without the extra newline. Then I have to split on '>', throw the first item in the list away, and add back the '>'. Arggh.
Another useful Python idiom is:
I always
2010 - b == b % 100 |
For today what it means is that I learned to type on a typewriter, an archaic device you can read about in wikipedia (here). My first two were unpowered. In typing class, my teacher was very strict: it was an error if you did not have two spaces separating sentences. In typography, or even when typing on a computer, the rules have changed, but legions of us still type to please the critical eye of someone like Brother McDermott.
Apparently that drives certain people crazy (here) or just leaves them bemused (here and here).
I still do it, as you can see it in the source for the blog. It's arguably correct for non-proportional fonts. But, in the end it doesn't matter because html simply ignores the extra space.
What does matter and I find incredibly useful is to separate groups of multi-line data by a double newline. Then I can just do:
L = data.strip().split('\n\n') |
It's mildly irritating when I send off a FASTA file in that format and it comes back without the extra newline. Then I have to split on '>', throw the first item in the list away, and add back the '>'. Arggh.
Another useful Python idiom is:
title, seq = item.strip().split('\n',1) |
I always
strip
, just to be safe.
Saturday, January 15, 2011
A series for pi
I saw something really nice in Strang (Calculus). It starts with inverse trigonometric functions, which are hard for me to think about, but let's go slowly and hope for the best:
x is the angle whose sin is y. We draw a picture, and see that:
and
since we're dealing with the inverse function
slope of inverse = 1 / slope of original function
Since
And our goal, remembering from here:
So, if we integrate
The geometric series is:
How do we derive this? One way (not legal) is to assume that the series really does converge to a sum S
Or we can check it by just multiplying out:
Replace x by -x2
If we integrate both sides, the rhs is
The magic: if we let
Another series for π, how cool is that?
y = sin x |
x is the angle whose sin is y. We draw a picture, and see that:
sin-1 y + cos-1 y = π/2 |
and
cos x = √ (1-y2) |
since we're dealing with the inverse function
sin-1 y
:slope of inverse = 1 / slope of original function
y = sin x |
Since
sin-1 y + cos-1 y = π/2
, if:x = cos-1 y |
And our goal, remembering from here:
(u/v)' = (v u' - u v')/v2 |
So, if we integrate
1/(1 + y2) dy
, we get tan-1 y
. We're going to use that.The geometric series is:
1 + x + x2 + x3 + .. = 1/(1 - x) |
How do we derive this? One way (not legal) is to assume that the series really does converge to a sum S
1 + x + x2 + .. = S |
Or we can check it by just multiplying out:
(1 - x) (1 + x + x2 + x3 + ..) = 1 |
Replace x by -x2
1 - x2 + x4 - x6 .. = 1/(1 + x2) |
If we integrate both sides, the rhs is
tan-1 x
. And the lhs is:x - x3/3 + x5/5 - x7/7 |
The magic: if we let
x = 1
, then:1 - 1/3 + 1/5 - 1/7 + .. = tan-1 1 = π/4 |
Another series for π, how cool is that?
Thursday, January 13, 2011
Finding Python (for PyObjC) 3
Just a short post to say that I asked about this on Stack Overflow and got a very nice answer from Ned Deily (here). He explains clearly what the different "executables" are about, and reminds me of a tool I'd overlooked:
otool
. For example> otool -L /System/Library/Frameworks/Python.framework/Versions/2.6/Resources/Python.app/Contents/MacOS/Python |
Although it is not so helpful here:
> otool -L /System/Library/Frameworks/Python.framework/Versions/2.6/bin/python |
I still think the
export DYLD_PRINT_LIBRARIES=1
trick is pretty cool. You can also look into
nm
(like here).I found
MachOView.app
at Source Forge (here) and tried it. It gave me quite a bit of insight into the structure of our simple examples from the other day (here). The screenshot above is from a run on its own executable. There is very detailed info on the object code (or images in geek-speak).I had been a bit worried about it since I can't find out anything about the guy (Peter Saghelyi). But I found the source, which is not under files. Do:
svn co https://machoview.svn.sourceforge.net/svnroot/machoview machoview |
Wednesday, January 12, 2011
Matt Gallagher, artist
I hope you find this as entertaining as I did. A Cocoa App (with a window, even) but not using Xcode). The App itself is a curiosity---though not so much of one as Amit Singh's even shorter program (see the link for details)---but Matt Gallagher obviously knows his stuff, and I can see it serving as a harness for various simple tests.
Maybe I should make a sidebar populated with blogs like his. Check it out.
Tuesday, January 11, 2011
Build Python on OS X with clang
I've been fooling around with
The date output:
2 minutes and 36 seconds!
The time stamp is interesting..
just hit it again and it'll finish
However, I was unable to do a framework build to this location, and I was unable to build matplotlib either, at least so far.
clang
as a gcc
replacement. (I'd love to have Xcode 4 but it's not worth the $99). Here we build Python 2.7 in record time:mkdir ~/temp |
The date output:
Wed Jan 12 15:35:26 EST 2011 |
cd Python-2.7.1/ |
Wed Jan 12 15:38:02 EST 2011 |
2 minutes and 36 seconds!
~/temp/Python/bin/python |
The time stamp is interesting..
> file ~/temp/Python/bin/python |
cd .. |
just hit it again and it'll finish
~/temp/Python/bin/python |
However, I was unable to do a framework build to this location, and I was unable to build matplotlib either, at least so far.
Sunday, January 9, 2011
Python for PyObjC
This is a short post to document that I was able to download the Python.org's Framework build of Python 2.7 and then install
The installer comes from here for
I did a stock install except I unchecked
Python.org installs stuff in
I added this to
I anticipated that
I couldn't find any decent docs, though. It looks like they've just copied the
The way my
pyobjc has lots of warnings but looks OK
Next, I installed
I changed it to Python 2.7.
I also needed to update to
As I said, the build instructions were as given at the link.
[UPDATE: Poking around in
I tested
Then, I got PyCogent (download link):
I had a few failures:
Some of these are related to executables not present. Looks pretty good. Finally, let's setup a new Xcode project:
Xcode > Cocoa-Python application
Since our Python is a universal binary with 64-bit:
Build the Xcode project as 64-bit as well. Add this to main.py:
And run it from the Console:
It works!
matplotlib
in it, as well as modify a Cocoa-Python application to link against that Framework. The only thing that looks a little shaky is the PyObjC install, but it still seems like it's working. Not every detail is given, but I hope it's enough.The installer comes from here for
python-2.7.1-macosx10.6.dmg
(Mac Installer disk image (2.7.1) for OS X 10.6 and later).I did a stock install except I unchecked
Shell profile updater
under Custom.. Not sure why, except I didn't want to spend time figuring out what it does.Python.org installs stuff in
/Library/Frameworks/Pythonframework |
I added this to
.bash_profile
> export PATH=/$PATH:/Library/Frameworks/Python.framework/Versions/2.7/bin |
I anticipated that
easy_install
would be difficult, but I got (something like) it from distribute
:> curl -O http://python-distribute.org/distribute_setup.py |
I couldn't find any decent docs, though. It looks like they've just copied the
easy_install
documentation which I found impossible to figure out so far. But, poking around, I found easy_install
and easy_install-2.7
in bin
:> which easy_install-2.7 |
The way my
$PATH
is set up, easy_install
runs the wrong Python. That's OK, we just do:> easy_install-2.7 -U numpy |
pyobjc has lots of warnings but looks OK
> /usr/local/bin/python |
Next, I installed
matplotlib
using instructions from Gavin Huttley (here, here). Since matplotlib-1.0.0
is available, I went for it. In the make.osx
file:PYVERSION=2.7 |
I changed it to Python 2.7.
I also needed to update to
libpng-1.2.39
and libfreetype-2.3.11
(although these are not the latest versions.. In fact, I had a bit of trouble with libpng
since I got back a file that was really html, though titled as .gz or .bz2 or .xz. I finally found it here). For the other one:> FREETYPEVERSION=2.3.11 |
As I said, the build instructions were as given at the link.
[UPDATE: Poking around in
libpng
file INSTALL
, I notice that it says: "Before installing libpng, you must first install zlib, if it is not already on your system." So I'll do it in that order next time.]I tested
matplotlib
by running the script from here.Then, I got PyCogent (download link):
> cd PyCogent-1.5 |
> cd tests |
I had a few failures:
Ran 3602 tests in 184.766s |
Some of these are related to executables not present. Looks pretty good. Finally, let's setup a new Xcode project:
Xcode > Cocoa-Python application
change SDK to 10.6
delete Python.framework
drag in new Python.framework
under Targets > X > Link Binary ..
drag in new Python.framework
Since our Python is a universal binary with 64-bit:
> /usr/local/bin/python |
Build the Xcode project as 64-bit as well. Add this to main.py:
import sys |
And run it from the Console:
2.7.1 (r271:86882M, Nov 30 2010, 10:35:34) |
It works!
Labels:
matplotlib,
PyObjC,
simple Python,
software installs,
Xcode
Saturday, January 8, 2011
More hidden treasure in Python
Saw another "easter egg" for the first time this morning (here).
It was tantalizing enough that I grabbed Python 2.7 from Macports:
But I don't want to spoil the secret completely.
It was tantalizing enough that I grabbed Python 2.7 from Macports:
> sudo port install python27 |
But I don't want to spoil the secret completely.
Hidden treasure in Python
Some fun Python history on this:
How was this ever found? It's not in
> python |
How was this ever found? It's not in
>>> dir(__builtins__)
Finding Python (for PyObjC) 2
One last post about PyObjC issues and finding which Python is running (under OS X) and then I'm done with it. Until I have another problem.. The structure of a framework is like this:
From the docs, a framework normally has a structure like that shown above, where (if A is the current version) the actual library would be:
The reason for the post is that I realized I've been overlooking something obvious: the existence of an
This key is not required to be set for a framework (a framework doesn't have to contain any code), but if it is, you'd expect it to be the path to the executable. Although there is something in the Framework docs about this key must be the same as the name of the framework. So the executable should be:
As far as I can see, our best approach is still to snoop on the loader. And to solve the original problem (no PyObjC) by making sure that we're running System Python.
MyFramework.framework/ |
From the docs, a framework normally has a structure like that shown above, where (if A is the current version) the actual library would be:
MyFramework.framework/Versions/A/MyFramework
The reason for the post is that I realized I've been overlooking something obvious: the existence of an
Info.plist
file. Just as we set the key: NSPrincipalClass
, to have the value: SimpleMessage
in our bundle (here), there is a key in the Python.framework:cat Resources/Info.plist |
This key is not required to be set for a framework (a framework doesn't have to contain any code), but if it is, you'd expect it to be the path to the executable. Although there is something in the Framework docs about this key must be the same as the name of the framework. So the executable should be:
/System/Library/Frameworks/Python.framework/Versions/2.5/Python
As far as I can see, our best approach is still to snoop on the loader. And to solve the original problem (no PyObjC) by making sure that we're running System Python.
Friday, January 7, 2011
Finding Python (for PyObjC)
This is a post exploring how to hunt down which Python is launched when we build and run a Python Cocoa application using Xcode and the templates from here.
The problem to be solved is that in some situations, the wrong Python is run, and it doesn't have PyObjC installed, so the statement
In order to fix that, we need to figure out which Python is running and either (i) change which one is run, (ii) remove the "wrong" one, (iii) install PyObjC into the "wrong" one or (iv) provide a path to PyObjC. I fixed my problem before by (ii). I never got (iii) working because I haven't figured out yet how to install
What we'll see here leads to the recommendation that you try (i).
We can start our troubleshooting by inserting this code into
Where does this come from? To begin with, it does not depend on this line in
The line can be commented out and everything still runs fine. I think it may be roadkill left over from the old days.
The second thing is that if we look at the Xcode build settings for either the project or the target we see:
Since the default SDK was set for 10.5, at link time we expect to link against:
If we double-click on the Python.framework (under Frameworks > Linked Frameworks in the Xcode project) we get a Finder window open to:
If we do get info with the Python.framework selected we get
but all of this says nothing about the version so one might expect it to go with Current, which is actually Python 2.6:
I've looked at environment info. Put this in
The only thing interesting is:
The
Let's try to find out more using this trick:
We're clearly loading the 2.5 framework. And this behavior can be changed by changing the SDK in Xcode. However, I can't show you anything more right now because I no longer have a failing example! (here)
I don't see anything in any of the settings for the Project, Target or Executable that would explain which Python is actually launched when the App runs. (As you'll see in a minute, there are two choices even for the Python.framework, for example in Version 2.5).
There is some weirdness about the system framework. On three different machines running OS X 10.6.5: Python at top-level in the 2.5 framework seems to be a dynamically linked library.
But it actually runs
The whole second half of the post, and the part just above, is a hideous mistake. See here for details.
Or as my boss likes to tell me, "some days you're the dog, and some days you're the hydrant"---he's amused, me not so much.
The problem to be solved is that in some situations, the wrong Python is run, and it doesn't have PyObjC installed, so the statement
import objc
fails with ImportError
, and the App terminates.In order to fix that, we need to figure out which Python is running and either (i) change which one is run, (ii) remove the "wrong" one, (iii) install PyObjC into the "wrong" one or (iv) provide a path to PyObjC. I fixed my problem before by (ii). I never got (iii) working because I haven't figured out yet how to install
easy_install
when it's not there yet, or alternatively build PyObjC from scratch. What we'll see here leads to the recommendation that you try (i).
We can start our troubleshooting by inserting this code into
main.py
before the import of objc
, as shown:import sys |
2.5.4 (r254:67916, Jun 24 2010, 21:47:25) |
Where does this come from? To begin with, it does not depend on this line in
main.m
:Py_SetProgramName("/usr/bin/python"); |
The line can be commented out and everything still runs fine. I think it may be roadkill left over from the old days.
The second thing is that if we look at the Xcode build settings for either the project or the target we see:
Base SDK Mac OS X 10.5
The name or path of the base SDK being used during the build. The product will be built against the headers and libraries located inside the indicated SDK. This path will be prepended to all search paths, and will be passed through the environment to the compiler and linker. Normally, this path is set at the project level via the "Cross-Develop Using Target SDK" popup in the General tab of the project inspector. Additional SDKs can be specified in the ADDITIONAL_SDKS setting. [SDKROOT]
Since the default SDK was set for 10.5, at link time we expect to link against:
/Developer/SDKs/MacOSX10.5.sdk/System/Library/Frameworks/Python.framework
If we double-click on the Python.framework (under Frameworks > Linked Frameworks in the Xcode project) we get a Finder window open to:
/System/Library/Frameworks/Python.framework
If we do get info with the Python.framework selected we get
Name: Python.framework
Path: /System/Library/Frameworks/Python.framework
but all of this says nothing about the version so one might expect it to go with Current, which is actually Python 2.6:
/System/Library/Frameworks/Python.framework/Versions/2.6
I've looked at environment info. Put this in
main.py
:import os |
The only thing interesting is:
PATH /Developer/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin |
The
PATH
variable is not the one from my shell. But there isn't any Python in /Developer/usr/bin
and both Python and python in /usr/bin
give 2.6. So that's not where it comes from. $PYTHONPATH just points to the App.Let's try to find out more using this trick:
> export DYLD_PRINT_LIBRARIES=1 |
We're clearly loading the 2.5 framework. And this behavior can be changed by changing the SDK in Xcode. However, I can't show you anything more right now because I no longer have a failing example! (here)
I don't see anything in any of the settings for the Project, Target or Executable that would explain which Python is actually launched when the App runs. (As you'll see in a minute, there are two choices even for the Python.framework, for example in Version 2.5).
There is some weirdness about the system framework. On three different machines running OS X 10.6.5: Python at top-level in the 2.5 framework seems to be a dynamically linked library.
But it actually runs
/usr/bin/Python
which is 2.6!:> cd /System/Library/Frameworks/Python.framework/Versions/2.5 |
The whole second half of the post, and the part just above, is a hideous mistake. See here for details.
The stupid, it burns.link
Or as my boss likes to tell me, "some days you're the dog, and some days you're the hydrant"---he's amused, me not so much.
Subscribe to:
Posts (Atom)