Swift framework from the command line

Following up on the previous post, I want to import my new Swift framework when building, or running Swift code from the command line.

The example above shows building. It works!

I struggled with this.. and now that I know, it just seems so silly. When providing the path to the framework, don't provide the full path, just give the path to the folder that contains it ... of course. We also need the path to the SDK, which Xcode will prompt you about, should you leave that part out.

For the other method (xcrun swift file.swift) this doesn't work. That's probably not too surprising, but maybe there will be a way.

gpg

This post is about the Gnu Privacy Guard (gpg). A link from the main page leads to a download of an installer for OS X here.

I followed the Quick start tutorial here, and sent an encrypted email between two of my accounts using Mail after just a few minutes. Nice!

The lock and signature icons in the lower right-hand corner are only active if the recipient's public key is available.

Let's explore a bit in Terminal
generate a new key with a spectacularly weak passphrase: abc

> gpg --gen-key gpg (GnuPG/MacGPG2) 2.0.18; Copyright (C) 2011 Free Software Foundation, Inc. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Please select what kind of key you want: (1) RSA and RSA (default) (2) DSA and Elgamal (3) DSA (sign only) (4) RSA (sign only) Your selection? 1 RSA keys may be between 1024 and 4096 bits long. What keysize do you want? (2048) Requested keysize is 2048 bits Please specify how long the key should be valid. 0 = key does not expire = key expires in n days w = key expires in n weeks m = key expires in n months y = key expires in n years Key is valid for? (0) 4y Key expires at Thu Apr 28 10:11:26 2016 EDT Is this correct? (y/N) y GnuPG needs to construct a user ID to identify your key. Real name: Alice Email address: alice@email.com Comment: You selected this USER-ID: "Alice " Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O You need a Passphrase to protect your secret key. We need to generate a lot of random bytes. It is a good idea to perform some other action (type on the keyboard, move the mouse, utilize the disks) during the prime generation; this gives the random number generator a better chance to gain enough entropy. gpg: key 6CC18DC3 marked as ultimately trusted public and secret key created and signed. gpg: checking the trustdb gpg: 3 marginal(s) needed, 1 complete(s) needed, PGP trust model gpg: depth: 0 valid: 4 signed: 0 trust: 0-, 0q, 0n, 0m, 0f, 4u gpg: next trustdb check due at 2015-08-18 pub 2048R/6CC18DC3 2012-04-29 [expires: 2016-04-28] Key fingerprint = A84B 4F1D A439 849F 47F1 55F9 BF7C FB80 6CC1 8DC3 uid Alice sub 2048R/DC9C92B3 2012-04-29 [expires: 2016-04-28]

I have a feeling that this fake user's data may have been sent to the key server, and that's not nice. But I expected to get a prompt related to that and I didn't get one.

Let's take a quick look at what we have:

> gpg -k Alice pub 2048R/6CC18DC3 2012-04-29 [expires: 2016-04-28] uid Alice sub 2048R/DC9C92B3 2012-04-29 [expires: 2016-04-28] > gpg -k --fingerprint Alice pub 2048R/6CC18DC3 2012-04-29 [expires: 2016-04-28] Key fingerprint = A84B 4F1D A439 849F 47F1 55F9 BF7C FB80 6CC1 8DC3 uid Alice sub 2048R/DC9C92B3 2012-04-29 [expires: 2016-04-28] >

-a is "ascii armored" base64 output:

> gpg -a --export 6CC18DC3 -----BEGIN PGP PUBLIC KEY BLOCK----- Version: GnuPG/MacGPG2 v2.0.18 (Darwin) Comment: GPGTools - http://gpgtools.org mQENBE+dTEUBCADWaa0ikPdHmp7OONsuhUJeIIr.. .. -----END PGP PUBLIC KEY BLOCK----- > > gpg -a --export-secret-key 6CC18DC3 -----BEGIN PGP PRIVATE KEY BLOCK----- Version: GnuPG/MacGPG2 v2.0.18 (Darwin) Comment: GPGTools - http://gpgtools.org lQO+BE+dTEUBCADWaa0ikPdHmp7OONsuhUJeIIr.. .. -----END PGP PRIVATE KEY BLOCK-----

---

While this looks quite similar to RSA (my posts), the Python rsa module won't handle it as is. I didn't dissect it by hand yet. There's something called pgpdump that is recommended on the web.

It builds easily and then I can do:

gpg --export 6CC18DC3 | pgpdump -i gpg --export 6CC18DC3-secret-key | pgpdump -i

but I didn't find the output all that useful yet. It shows multiple values for n and e but doesn't show d or p and q.

---

Try some encryption:

> gpg -e -r 6CC18DC3 m.txt > hexdump -C m.txt 00000000 68 65 6c 6c 6f 2c 20 77 6f 72 6c 64 21 0a |hello, world!.| 0000000e > hexdump -C m.txt.gpg 00000000 85 01 0c 03 a9 47 9a 63 dc 9c 92 b3 01 07 ff 57 |.....G.c.......W| 00000010 86 9b 5d a0 40 d1 f0 ef 5a 6f dc eb 19 a9 eb 8c |..].@...Zo......| 00000020 ee 66 a7 34 84 e4 47 5b 6c 48 9f 9e 89 13 4c 2a |.f.4..G[lH....L*| 00000030 71 6a 31 b7 27 23 9d 56 a7 c2 ad fd db 47 57 68 |qj1.'#.V.....GWh| 00000040 da 75 9a 2d 2f f6 46 60 16 84 b6 17 bf e7 b7 5c |.u.-/.F`.......\| 00000050 36 fd d1 e2 22 ee 93 dc ad 82 f5 f1 46 99 12 f3 |6...".......F...| 00000060 fe 25 a1 b3 01 8c 37 a0 59 da ac 39 90 a4 1c ba |.%....7.Y..9....| 00000070 a0 4f 1e b6 da d5 36 55 b1 17 d6 c4 5a 28 de b4 |.O....6U....Z(..| 00000080 47 b2 af 8a c8 9c 58 85 44 f8 08 fe a1 47 c3 8f |G.....X.D....G..| 00000090 4d b1 78 50 87 dc a7 7f 55 89 f2 6e 7f 75 ae a0 |M.xP....U..n.u..| 000000a0 69 68 46 5a 64 1e b4 6e c7 ee 84 77 8d a4 ce 14 |ihFZd..n...w....| 000000b0 72 45 13 be d0 33 5c d6 23 6f 2d b2 84 2f d9 55 |rE...3\.#o-../.U| 000000c0 f7 de d2 8f b6 20 5b 71 4e 31 ae b8 d7 1b 09 bf |..... [qN1......| 000000d0 80 9e e0 1f 47 cb 73 a1 59 42 81 24 1f 2b de 4b |....G.s.YB.$.+.K| 000000e0 0d 23 fc c6 a2 83 5e c2 b3 e5 9f 1f 32 ae 75 07 |.#....^.....2.u.| 000000f0 79 7f 51 49 02 80 a8 47 c4 5c b6 6f aa ac d4 5c |y.QI...G.\.o...\| 00000100 e7 c9 b6 1f d2 c1 7e 03 45 34 59 85 d1 63 01 d2 |......~.E4Y..c..| 00000110 4e 01 27 2e e9 09 aa 82 5d 77 56 82 22 4e 2e 67 |N.'.....]wV."N.g| 00000120 1c 4a bc ba c1 43 d6 f0 86 02 5d e7 b3 58 74 79 |.J...C....]..Xty| 00000130 bc 15 69 d4 44 ba f6 76 0c a7 a1 d5 9b 1b e0 8b |..i.D..v........| 00000140 b6 7b df db b0 5f e6 34 0b 36 14 0b fd c6 62 f3 |.{..._.4.6....b.| 00000150 16 a8 97 ad 92 e7 4e a4 ee ab 59 53 91 c6 52 |......N...YS..R| 0000015f > gpg -d -o p.txt m.txt.gpg You need a passphrase to unlock the secret key for user: "Alice " 2048-bit RSA key, ID DC9C92B3, created 2012-04-29 (main key ID 6CC18DC3) gpg: encrypted with 2048-bit RSA key, ID DC9C92B3, created 2012-04-29 "Alice " > cat p.txt hello, world!

I found what looks to be an excellent tutorial on the web. I still need to work through it.

mtree on OS X

I came across a utility to test for changes in files (and more stuff, like permissions) in Chapter 4 of EdgeEnterprise Mac Security: mtree.

I'd never heard of it. There's a writeup about it here, as well as a couple of blog entries (here and here)

Rather than go straight to mtree (I still have to read the man page), this is a write-up of the bash script posted on the the first blog post. So it's really more about improving my bash scripting-fu (origin here) than it is about mtree.

> nl script.sh 1 #!/bin/sh 2 SCRIPT="`basename $0 | sed 's/\..*$//'`" 3 CONFIG=${1:-$HOME/.$SCRIPT} 4 DIR=${2:-$HOME/Documents} 5 cd $DIR || \ 6 { echo "Failed to change working directory to $DIR" >&2; exit 1; } 7 if [ -f $CONFIG ]; then 8 /usr/sbin/mtree -f $CONFIG 9 fi 10 /usr/sbin/mtree -c -k time,md5digest,ripemd160digest > $CONFIG>

I added line numbers with nl, you could use cat or other methods.

I hope line 1 is obvious (see wikipedia if it's not).

In line two, we take the variable $0, which is the path to the script, obtain its basename, and then strip off the extension:

> basename /Users/telliott/script.sh script.sh > echo x.txt | sed 's/\..*$//' x

I had an even harder time with lines 3 and 4, but the folks on StackOverflow helped me.

I learned that bash has a man page (of course!). Also that one can search within a man page by entering "/ :-" after you get the page, which initiates a search for ":-". The answer:

${parameter:-word}
Use Default Values. If parameter is unset or null, the expansion
of word is substituted. Otherwise, the value of parameter is sub-
stituted.

So these two lines

3 CONFIG=${1:-$HOME/.$SCRIPT} 4 DIR=${2:-$HOME/Documents}

simply give default values for $CONFIG and $DIR if we don't supply them as arguments ($1 and $2) when invoking the script. If we called the script by ./script.sh from the Desktop, then we'd have $CONFIG be equal to ~/.script (a hidden file) and $DIR be equal to ~/Documents.

Lines 5 and 6 change the working directory to ~/Documents as default, if this fails (because the path input as an argument is no good) we write a message to >&2 and exit with 1 signifying failure. >&2 is stderr. This writes to the Terminal for me.

Lines 7 through 9 check whether $CONFIG exists already, if it does then we call mtree with the -f flag and $CONFIG argument:

/usr/sbin/mtree -f $CONFIG

From the mtree man page:

-f file
Read the specification from file, instead of from the standard
input.

If this option is specified twice, the two specifications are com-
pared with each other, rather than to the file hierarchy. The
specifications be sorted like output generated using -c. The out-
put format in this case is somewhat remniscent of comm(1), having
"in first spec only", "in second spec only", and "different" col-
umns, prefixed by zero, one and two TAB characters respectively.
Each entry in the "different" column occupies two lines, one from
each specification.

But the first time through, $CONFIG shouldn't exist yet. Instead, we call mtree with the -c and -k flags followed by various arguments, and write the result to $CONFIG. Let's do it in a simple way, as shown below. From the Desktop directory I input the command as shown, and write the results to x.txt in $HOME. As you can see, timestamps for all the Desktop files are there:

/usr/sbin/mtree -c -k time > x.txt > cat ~/x.txt # user: telliott_admin # machine: Toms-Mac-mini.local # tree: /Users/telliott_admin/Desktop # date: Mon Apr 23 17:20:21 2012 # . /set type=file . type=dir time=1335216020.0 .DS_Store time=1335216014.0 .localized time=1333333575.0 notes.txt time=1335210868.0 post.txt time=1335216020.0 script.sh time=1335215494.0 script.sh.txt \ time=1335211653.0 todo.txt time=1335204803.0 .. >

Now, if I do it with -f, I get:

> /usr/sbin/mtree -f ~/x.txt . changed modification time expected Mon Apr 23 17:20:20 2012 found Mon Apr 23 17:20:51 2012 .DS_Store changed modification time expected Mon Apr 23 17:20:14 2012 found Mon Apr 23 17:20:55 2012 post.txt changed modification time expected Mon Apr 23 17:20:20 2012 found Mon Apr 23 17:20:51 2012

I see a change because I'm writing this text into post.txt! For some reason .DS_Store also changed. Next time I'll try to do something a little more useful.

Unix spoken here (4)

There are a just few more Unix commands I should mention before we exhaust everyone's patience. (Three posts here, here, and here).

I'm sure you're familiar with sudo, which stands for superuser, do once. If not, you can see wikipedia. Normally you will use this to write (install) or even delete files in /usr/bin and the like.

> mv /usr/bin/znew ~/Desktop/znew mv: rename /usr/bin/znew to /Users/telliott_admin/Desktop/znew: Permission denied

Don't forget to read the man page on this one. There is always something interesting in the manual.

Many Unix commands appear simple but have numerous options and arguments that make them more like a Swiss Army Knife than a pocketknife. Output on the command line using man can be broken up into chunks or streamed using the down arrow, but maybe you want to study it or keep it as a reference.

For some reason you need to format the output from man in a special way before directing it to a file, for example:

man ls | col -bx > results.txt head -n 8 results.txt .. SYNOPSIS ls [-ABCFGHLOPRSTUW@abcdefghiklmnopqrstuwx1] [file ...]

Nearly the whole alphabet! But doing this without the col leads to selective stuttering:

LS(1) BSD General Commands Manual LS(1) N NA AM ME E l ls s -- list directory contents S SY YN NO OP PS SI IS S l ls s [- -A AB BC CF FG GH HL LO OP PR RS ST TU UW W@ @a ab bc cd de ef fg gh hi ik kl lm mn no op pq qr rs st tu uw wx x1 1] [_ f_ i_ l_ e _ ._ ._ .] D DE ES SC CR RI IP PT TI IO ON N For each operand that names a _ f_ i_ l_ e of a type other than directory, l ls s

head and tail are useful for looking at the first few lines or last few lines of a file. less is like what you get when you just do man some_command ---it gives chunks or streams.

Try man cat to see what else that can do.

There is also split but I couldn't figure out how to make it take my favorite separator ('\n\n').

diff does what you might expect. The use case might be that you have some good edits but also some bad edits to a file, so you want to revert to the old one, and then try the edits one by one.

> diff old.txt new.txt > results.txt > cat results.txt 28a29 > my edit 50,51c51 < < --- > deleted a line here

Or drop results.txt onto TextMate for a little color (graphic).



There's a command to find out information about different processes with finer grained control than Activity Viewer. For example:

ps -u username ps -U userid ps -p processid

Say:

> ps -p 48068 -O %mem PID %MEM TT STAT TIME COMMAND 48068 1.1 ?? S 0:00.91 /Applications/R64.app/Contents/MacOS/R -psn_0_2404939

The man page on ps is extensive.

find is tricky. I need a lot more practice to understand it. Here's a simple one, and then a fun one though:

> find . -name "old.txt" -print ./old.txt > find -f ~/* | wc 118791 235608 11536780

I "find" all the files under my home directory and feed the results to wc (word count), discovering that there are 118,791 lines in the output---that's the number of files! This link seems to have good info on find.

Sometimes I just want to use Python as a calculator or get an ascii value, but without calling up the interpreter (and having to dismiss it):

> python -c 'print 3.0/17' 0.176470588235 > python -c 'print ord("a")' 97 > python -c 'print chr(101)' e > python -c 'import math; print math.e' 2.71828182846

> hexdump -C post.txt 00000000 4d 61 6e 79 20 55 6e 69 78 20 63 6f 6d 6d 61 6e |Many Unix comman| 00000010 64 73 20 61 70 70 65 61 72 20 74 6f 20 62 65 20 |ds appear to be | 00000020 73 69 6d 70 6c 65 20 62 75 74 20 68 61 76 65 20 |simple but have |

Finally, here are a few more that either I've found useful or I'm sure they will be, but I don't feel expert enough to try to explain them. Have fun exploring:

alias awk curl hash md5 openssl sha1 set tar top touch unset

If you get bored here; or try to check out how Greg's email address gets generated!

And there's always ssh. But that really does deserve a post of its own.

Unix spoken here (3)

Let's do some more with Unix commands. Previous posts here and here.

You might have noticed that I have a very short prompt in Terminal, just the character `>` followed by a space. That's because I have an (invisible) file in my home directory named .bash_profile, containing this line:

> cat ~/.bash_profile PS1="> "; export PS1

We can change back to the original behavior by changing the name of the file or moving it, and then re-starting Terminal:

mv ~/.bash_profile ~/Desktop/x.txt

c-98-236-78-154:Desktop telliott$

We could comment out that line of the file (now that it's visible with no leading `.` in the filename, we can easily open it in TextEdit).

#PS1="> "; export PS1

But I like it, so I'll revert to the original situation.

I think there is a way to get TextEdit to show hidden files in the Open dialog, but I forget how at the moment. In any event, it's interesting that having opened and closed the file while it was visible on the Desktop, Open Recent now shows .bash_profile as an option.

Another way to edit the file is to use pico (a command line editor):

> pico ~/.bash_profile

I'm sure you can figure out how to use pico.

One more useful thing in .bash_profile is modification to the $PATH variable. Let's look at that from the command line:

> echo $PATH /opt/local/bin:/opt/local/sbin:/Users/telliott/bin/:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/ncbi/blast/bin:/usr/texbin:/usr/X11/bin

When I call ls, for example, these directories are searched in turn until the program is discovered finally. It is in:

> whereis ls /bin/ls

If I wanted to modify the $PATH variable I could do:

> PATH=~/Desktop:$PATH; export PATH > echo $PATH /Users/telliott/Desktop:/opt/local/bin:/opt/local/sbin:/Users/telliott/bin/:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/local/ncbi/blast/bin:/usr/texbin:/usr/X11/bin

My $PATH is pretty complicated for several reasons: use of MacPorts, some things related to ncbi and X11, etc. It would take too long to explain.

The shell has command history. If you want to repeat a command, just press the up arrow key. To repeat a command further back, keep pushing until you see the one you want. Then press Return. Suppose you have a series of commands (say 3 in a row), and the first of these is 6 commands ago. Press the arrow 6 times, then return. Press the arrow 6 times, then return. Press the arrow 6 times, then return. Because the command history is updated each time, 6 arrow presses gets you back exactly where you need to be for each of the three commands in the sequence.

Alternatively you could do history:

> history 501 cd Desktop 502 pico ~/bash_profile 503 echo $PATH 504 file ls 505 whereis ls 506 PATH=~/Desktop:$PATH; export $PATH 507 PATH=~/Desktop:$PATH; export PATH 508 echo $PATH 509 history

which lists each command I've entered since the last time I booted my machine (even saves between Terminal sessions). One can select a command from the history by number or by partial name match:

> !505 whereis ls /bin/ls > !wh whereis ls /bin/ls

First it prints the command, and then the result. You can combine commands on a single line with `;`

> !504; !512 whereis ls; echo abc /bin/ls abc > history | tail -2 515 whereis ls; echo abc 516 history > !515 whereis ls; echo abc /bin/ls abc

We're getting pretty good with this stuff! We don't really want to list all 500+ commands in the history, just the last two. So we feed the results of history to the Unix tool called tail through a `|`, a Unix "pipe." tail takes an option for how many lines at the end of the "file" to display.

The Unix command grep searches lines of text for those matching a pattern. It's particularly useful in combination with `|`.

> history | grep pico 40 pico ~/.hgrc 298 pico 502 pico ~/bash_profile 515 history | grep pico

Here, I edited some file with pico but I can't remember which one it was. So the entire output of the history is fed through the pipe to grep, searching for lines containing the pattern. Now I can just do:

> !40 pico ~/.hgrc

I won't show you the file.

The last command for this post is ln with the option -s which makes a "soft link" to a file. I can never remember, but you want to put the existing file first (that's the opposite of the way I would design it):

> cat temp/y.txt echo "abc" > ln -s foo temp/y.txt ln: temp/y.txt: File exists > ln -s temp/y.txt foo > ls foo temp > cat foo echo "abc" > ls -l total 8 lrwxr-xr-x 1 telliott staff 10 Jan 31 07:53 foo -> temp/y.txt drwxr-xr-x 5 telliott staff 170 Jan 31 07:53 temp

Now the listing for foo shows an `l` as the first character in the file mode, and it also shows -> temp/y.txt, where it links to.

Unix spoken here (2)

Let's continue with an introduction to Unix commands. First post here. And here are two more good links (one, two).

Let's start by putting some data into a file. We can use the command echo and the `>` redirection operator. Take a look at what's there with cat:

> cd > cd Desktop > echo "abc" > x.txt > cat x.txt abc

Suppose we wish to move this file to a new sub-directory. First we construct the new directory and then we move the file with mv:

> mkdir temp > mv x.txt temp/x.txt > ls temp x.txt > ls temp

Let's make the file `x.txt` actually do something, e.g.:

> echo "echo xyz" > temp/x.txt > cat temp/x.txt echo xyz > sh temp/x.txt xyz

Note that we wrote over the previous contents of temp/x.txt and the file system did not warn us first. There is no way to recover from this if it was a mistake!

The sh command executes the text in the file you give it; in this case the output is familiar. Normally, we'd probably use the file extension .sh for such a script, but it's not required. (Even with Python scripts, the .py extension is only required if you want to import a module).

We can also use the she-bang thing (#!) to make the code in a file executable. Use TextEdit to save the following code in a file y.txt on the Desktop:

#! sh echo abc

Then we do the following. An executable can be executed just by entering its path, but.. there is a restriction that you can't do that for a program in the current directory. You must specify it as the relative path from the directory where we are currently./executable where the `.` means this directory (the restriction has never made a lot of sense to me either, but it's apparently a security hole):

> cat y.txt #! sh echo abc > ./y.txt -bash: ./y.txt: Permission denied

What has happened? We need to make the file "executable" by setting the "permission bits." Without getting into too much detail, these are output using a new option with the ls command:

> ls -l y.txt -rw-r--r-- 1 telliott staff 19 Jan 30 20:08 y.txt > chmod 711 y.txt > ls -l y.txt -rwx--x--x 1 telliott staff 19 Jan 30 20:08 y.txt > ./y.txt abc

The three permissions are r (read), w (write) and x (execute), listed in order for the user, her group, and the world. 7 is all of them. `x` is 1, `w` is 2, `r` is 4, and the combinations result from addition.

There are various shorthands for this. First reverting the previous change, then we could do this:

> chmod 644 y.txt > chmod u+x y.txt > ls -l y.txt -rwxr--r-- 1 telliott staff 19 Jan 30 20:08 y.txt

We have given the user (only) permission to execute the file.

> ls -l ~/Desktop total 1 drwxr-xr-x 2 telliott staff 102 Jan 30 20:08 temp -rw-r--r-- 1 telliott staff 15 Jan 30 20:08 y.txt

When we do ls on the Desktop directory (above), we get a listing for both the temp sub-directory and the file. Note that the first character in the "file mode" is d for the directory. staff is my group name, and after that comes the number of bytes in each "file", including directories.

From man ls

The Long Format
If the -l option is given, the following informa-
tion is displayed for each file: file mode, num-
ber of links, owner name, group name, number of
bytes in the file, abbreviated month, day-of-
month file was last modified, hour file last mod-
ified, minute file last modified, and the path-
name. In addition, for each directory whose con-
tents are displayed, the total number of 512-byte
blocks used by the files in the directory is dis-
played on a line by itself, immediately before
the information for the files in the directory.
If the file or directory has extended attributes,
the permissions field printed by the -l option is
followed by a '@' character. Otherwise, if the
file or directory has extended security informa-
tion (such as an access control list), the per-
missions field printed by the -l option is fol-
lowed by a '+' character.

And finally, ls also has a recursive option, which can be grouped into -lR:

> ls -lR total 8 drwxr-xr-x 3 telliott_admin staff 102 Jan 31 06:59 temp -rw-r--r-- 1 telliott_admin staff 15 Jan 31 07:00 y.txt ./temp: total 8 -rw-r--r-- 1 telliott_admin staff 4 Jan 31 06:59 x.txt

Let's continue by copying this file into our temp directory:

> cp y.txt temp > ls temp y.txt > ls temp x.txt y.txt

On the other hand, mv actually "moves" the file:

> mv y.txt temp > ls temp > ls temp x.txt y.txt

Once again, note the lack of warnings. The old copies of y.txt on the Desktop (and in temp) are gone forever.

Eventually, we tire of this. First we try to delete a single file:

> rm temp/y.txt > ls temp x.txt

But maybe temp has lots of files, so we try:

> rmdir temp rmdir: temp: Directory not empty > rm -r temp > ls >

The `-r` option is really dangerous. The classic Unix "joke" is to tell someone to do some combination of rm and -r and *. Don't fall for it.

You might think about looking at the man page for rm and maybe using the -i option.

Unix spoken here (1)

There are probably hundreds of pages on the web with good info about basic Unix commands (e.g. here, here). I thought I'd give some of the commands I think are most useful, but I have to start at the beginning. So, here goes with Unix in 5 minutes, Unleashed, For Dummies..

> cd > pwd /Users/telliott > cd Desktop > pwd /Users/telliott/Desktop > cd ~ > pwd /Users/telliott > cd Desktop > ls x.txt > cat x.txt some data

The first command, cd, says to change directory; with no other argument given, we go to our home directory. To see where we are in the file system we can do pwd: print working directory. Also, the ~ symbol is a shorthand for the user's home directory.

The "shell" is smart and has "command-completion." For example, from my home directory I enter cd De and then press tab. The shell fills out "Desktop."

Another very common command is ls: list the contents of the current directory (since there is no argument). And to see the contents of a file you might do cat with the results shown.

> ls -a . .DS_Store .temp .. .localized x.txt > ls /usr/bin/pyth* /usr/bin/python /usr/bin/python2.6-config /usr/bin/python-config /usr/bin/pythonw /usr/bin/python2.5 /usr/bin/pythonw2.5 /usr/bin/python2.5-config /usr/bin/pythonw2.6 /usr/bin/python2.6

Any directory may have "hidden" files whose names are prefaced by a dot `.`. These can be shown with ls -a. There are many (many) other options with results cataloged under man some_command.

The shell understands paths to files in both an absolute and relative sense. A path starting with / is absolute, otherwise it's relative. Hence /usr/bin is a directory starting from the root of the file system, while Desktop is only understood as a valid destination if I'm in my home directory (or some other directory with Desktop as a sub-directory).

The character * is a wild card and matches anything; ls /usr/bin/pyth* lists all the entries matching pyth- + something in the given directory.

> pushd /usr/bin /usr/bin ~/Desktop > pwd /usr/bin > popd ~/Desktop > pwd /Users/telliott/Desktop > cat -n ~/Desktop/x.txt 1 some data > cal January 2011 Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 > cd .. > pwd /Users/telliott_admin

Sometimes we want to make a jump in the file system but intend to come right back. pushd and popd are useful here, with results as shown.

As one more example of using options or arguments to a command, cat takes the option -n, which numbers the output lines. cal is probably faster than finding the Calendar.app, since I don't usually have it in the Dock.

Finally, to move toward the tips of the file system tree, you specify the directory at the next level so it has to be explicit, but there is only one parent, which we can reach implicitly with cd .., meaning "go up."

XCode template instructions archived

After I posted a link to the instructions I found on the web for getting Ronald Oussoren's XCode templates into XCode 3.25 (my post here), I went back for another look today. But I found that the page has been "disappeared" off the web. Luckily, Google still had a cache of it (I'm not sure how long that will last?). Anyway, here is what I did:

svn co http://svn.red-bean.com/pyobjc/trunk/pyobjc/pyobjc-xcode cd pyobjc-xcode/Project\ Templates/ ./project-tool.py -k -v --template Cocoa-Python\ Application/CocoaApp.xcodeproj/TemplateInfo.plist Cocoa-Python\ Application ~/Library/Application\ Support/Developer/Shared/Xcode/Project\ Templates/CocoaPython/Cocoa-Python\ Application ./project-tool.py -k -v --template Cocoa-Python\ Document-based\ Application/CocoaDocApp.xcodeproj/TemplateInfo.plist Cocoa-Python\ Document-based\ Application/ ~/Library/Application\ Support/Developer/Shared/Xcode/Project\ Templates/CocoaPython/Cocoa-Python\ Document-based\ Application ./project-tool.py -k -v --template Cocoa-Python\ Core\ Data\ Application/CocoaApp.xcodeproj/TemplateInfo.plist Cocoa-Python\ Core\ Data\ Application/ ~/Library/Application\ Support/Developer/Shared/Xcode/Project\ Templates/CocoaPython/Cocoa-Python\ Core\ Data\ Application ./project-tool.py -k -v --template Cocoa-Python\ Core\ Data\ Document-based\ Application/CocoaDocApp.xcodeproj/TemplateInfo.plist Cocoa-Python\ Core\ Data\ Document-based\ Application/ ~/Library/Application\ Support/Developer/Shared/Xcode/Project\ Templates/CocoaPython/Cocoa-Python\ Core\ Data\ Document-based\ Application

(I put some newlines in at the forward slashes, and tested one, if I screwed it up let me know).

[ UPDATE: Turns out I did screw it up, and no one let me know! The internet is a tough world---no second chances! ]

As the instructions said, once you restart XCode, you should see the four Cocoa-Python templates appear under “User Templates” like this:



Now do this:

cd ../File\ Templates/Cocoa/

And if it doesn’t already exist, create a File Templates directory in the current user’s library folder as follows:

mkdir ~/Library/Application\ Support/Developer/Shared/Xcode/File\ Templates cp -r * ~/Library/Application\ Support/Developer/Shared/Xcode/File\

That's it.

User's environment in OS X

The question arises: how to get access to the user's path on OS X. Stack Overflow does not disappoint (here):

// gcc -o test test.m -framework Foundation -fobjc-gc-only // .test #import <Foundation/Foundation.h> int main (int argc, const char* argv[]) { NSDictionary *eD = [[NSProcessInfo processInfo] environment]; NSLog(@"%@",[eD description]); return 0; }

$ ./test 2010-12-14 11:56:23.002 test[650:903] { "Apple_PubSub_Socket_Render" = "/tmp/launch-bvNjPH/Render"; "COMMAND_MODE" = unix2003; DISPLAY = "/tmp/launch-cWN8zD/org.x:0"; HOME = "/Users/telliott_admin"; LANG = "en_US.UTF-8"; LOGNAME = "telliott_admin"; OLDPWD = "/Users/telliott_admin"; PATH = "/Users/telliott_admin/bin/:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin"; PWD = "/Users/telliott_admin/Desktop"; SHELL = "/bin/bash"; SHLVL = 1; TERM = "xterm-color"; "TERM_PROGRAM" = "Apple_Terminal"; "TERM_PROGRAM_VERSION" = 273; TMPDIR = "/var/folders/rK/rK4-yxtZHUGADBsC5YPRk++++TI/-Tmp-/"; USER = "telliott_admin"; "_" = "./test"; "__CF_USER_TEXT_ENCODING" = "0x1F5:0:0"; }

Merge PDFs in OS X

I wanted to join a bunch of single page scanned tiffs into a one giant pdf. It's a chapter from a book (Shssssh!) I used Preview to save them as individual pdf files and then thought, how to merge them? I saw this hint for OS X 10.5, but it doesn't work for me on 10.6.

Then I came across another link that suggested I use pdftk (pdf toolkit). It's described here, but I got it using my new MacPorts install.

port search pdftk sudo port install pdftk

The usage is

pdftk file1.pdf file2.pdf cat output outfile.pdf

It worked great. The only problem is the files were named: Scan1.pdf, Scan2.pdf, etc. and when sorted the order is Scan1.pdf, Scan10.pdf, etc., so they were out of order in the merge. I adopted a embarassingly inelegant solution. I shortened the names and then entered this command:

pdftk 0.pdf 1.pdf 2.pdf 3.pdf 4.pdf 5.pdf 6.pdf 7.pdf 8.pdf 9.pdf 10.pdf 11.pdf 12.pdf 13.pdf 14.pdf 15.pdf 16.pdf cat output Ch2.1.pdf

Looks great!

Timeit from the command line in Python

Python has a module called timeit, whose function should be obvious. It can be used simply from the command line.

python -m timeit [-n N] [-r N] [-s S] [-t] [-c] [-h] [statement ...]

The -s flags an argument that should be executed only once. Multiple -s [setup code] lines can be included. There can also be multiple statements which is, or are, executed many times and the timing reported. I was curious about the relative speed of two approaches to grouping list elements (post here). I put the code for the two functions into a module test.py (the listing is at the end of the post).

Here is what I got:

$ python -m timeit -s 'import test' 'list(test.grouper(2,"abcdef"))' 100000 loops, best of 3: 5.34 usec per loop $ python -m timeit -s 'import test' 'test.grouper(2,"abcdef")' 100000 loops, best of 3: 2.19 usec per loop $ python -m timeit -s 'import test' 'list(test.chunks(2,"abcdef"))' 100000 loops, best of 3: 2.85 usec per loop $ python -m timeit -s 'import test' 'test.chunks(2,"abcdef")' 1000000 loops, best of 3: 0.685 usec per loop

The itertools solution takes 2-3 times as long, and getting the actual list rather than a generator or iterator has overhead as well.

from itertools import izip_longest def grouper(n, iterable, fillvalue=None): global itertools "grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx" args = [iter(iterable)] * n return izip_longest(fillvalue=fillvalue, *args) def chunks(n, L): """ Yield successive n-sized chunks from L. """ for i in xrange(0, len(L), n): yield L[i:i+n]

NSURL to refer to a file path

I made a fairly simple Cocoa project (a checkbook application) that I would like to have its own icon. And specifically, I'd like to mark files so that the file system knows they belong to my app, are displayed with its icon, and launch the app when you double-click them. However, because it's so simple, this version of the project is not yet a document-based app.

Since my attempts haven't been fruitful so far, I decided to explore Launch Services, which the Finder uses to do these things. To do that, I need to make a file that I know I can load into my application. In this post, I show two ways to accomplish this very simple first step: one way is with an NString that is a path to the file, and one using NSURL. As the docs say: "NSURL objects can be used to refer to files, and are the preferred way to do so."

To make things simple, I am building and running from Terminal. The file is a text file containing "my data", with the .txt extension. Since its straight text, we can use either encoding as shown in the listing:

encoding:NSASCIIStringEncoding
encoding:NSUTF8StringEncoding

If you want garbage collection, see here.

As step 2 in the exploration, I changed the extension to be testuff. (Do this in the information window, otherwise the Finder will sometimes just hide the extension). My test application will still open the file correctly, if the extension is changed in the path.

If I now set "Open with:" to TextEdit or Smultron, it works properly. So the question is: what do these applications need to implement in order to respond properly to the double-click? What message does Launch Services send them?



The output from step 1 as displayed by NSLog (logging info removed):

fn = /Users/te/Desktop/x.txt s1 = my data URL = file://localhost/Users/te/Desktop/x.txt s2 = my data

And the code listing:

// gcc -o test test.m -framework Foundation // ./test #import <Foundation/Foundation.h> int main (int argc, const char * argv[]) { NSAutoreleasePool *p; p = [NSAutoreleasePool new]; NSString *s; NSString *fn = NSHomeDirectory(); fn = [fn stringByAppendingString: @"/Desktop/x.txt"]; NSError *e; s = [NSString stringWithContentsOfFile:fn encoding:NSASCIIStringEncoding error:&e]; NSLog(@"fn = %@", fn); NSLog(@"s1 = %@", s); NSURL *fu; fu = [NSURL fileURLWithPath:fn]; s = [NSString stringWithContentsOfURL:fu encoding:NSASCIIStringEncoding //encoding:NSUTF8StringEncoding error:&e]; NSLog(@"URL = %@", fu); NSLog(@"s2 = %@", s); [p drain]; return 0; }

Category on NSString: lstrip and rstrip

I posted recently about a strip method for NSString. Since the method "trims" all occurrences of a given character, it is like Python's strip(). I thought it would be interesting to implement lstrip() and rstrip(). I did this as a category on NSString. Here is another NSString category that has more to do with Bioinformatics.

In the example, the app is built from the command line (with garbage collection), and all the code is in a single file. You will probably want to use XCode.

This is the output:

xyz 7 xyz 5 xyz 5

And here is the code:

// gcc -o test test.m -framework Foundation -fobjc-gc-only // ./test #import <Foundation/Foundation.h> @interface NSString (Stripper) -(NSString *) lstrip; -(NSString *) rstrip; @end @implementation NSString(Stripper) - (NSString *) lstrip { int i, N; unichar c; N = [self length]; for (i = 0; i < N; i++) { c = [self characterAtIndex:i]; if (!(c == ' ')) { break; } } if (i == N) { return nil; } return [self substringFromIndex:i]; } - (NSString *) rstrip { int i, N; unichar c; N = [self length]; for (i = N-1; i > -1; i--) { c = [self characterAtIndex:i]; if (!(c == ' ')) { break; } } if (i == -1) { return nil; } return [self substringToIndex:i+1]; } @end int main (int argc, const char * argv[]) { NSString *s = [NSString stringWithString:@" xyz "]; NSLog(@"%@ %i", s, [s length]); NSString *s2 = [s lstrip]; NSLog(@"%@ %i", s2, [s2 length]); NSString *s3 = [s rstrip]; NSLog(@"%@ %i", s3, [s3 length]); return 0; }

Can I get a date?

NSDate

I'm pretty sure this is not the official way to do it, but an easy way to extract the year-month-date information from an NSDate instance is to do

componentsSeparatedByString:@" "

on the description. Also, notice that constructing an NSDate object from a string requires a complete specification---including the time zone!

And at first I thought, there seems to be a bug in Apple's implementation of the dateWithString method. We specified January 1, but the resulting NSDate is one day (and one year) earlier. The answer lies in the time zone. Cocoa has converted my GMT time in constructing the NSDate to the current time zone where I am, even correcting for daylight savings time on the appropriate dates.

today 2009-09-30 16:05:32 -0400 array a ( "2009-09-30", "16:05:32", "-0400" ) array b ( 2009, 09, 30 ) birthday 2000-12-31 19:00:00 -0500 present age (seconds) 276033932.002803 age doubled 2018-06-30 12:11:04 -0400 age halved 2005-05-17 06:02:46 -0400

// gcc -o test test3.m -framework Foundation -fobjc-gc-only // ./test #import <Foundation/Foundation.h> int main (int argc, const char * argv[]) { //NSCalendarDate is deprecated NSDate *today = [NSDate date]; NSLog(@"today %@", [today description]); NSArray *a = [[today description] componentsSeparatedByString:@" "]; NSArray *b = [[a objectAtIndex:0] componentsSeparatedByString:@"-"]; NSLog(@"array a %@", a); NSLog(@"array b %@", b); // creating a date from a string // requires all the pieces! NSString *s = @"2001-01-01 00:00:00 +0000"; NSDate *bd = [NSDate dateWithString:s]; NSLog(@"birthday %@", [bd description]); NSTimeInterval ti = [today timeIntervalSinceDate:bd]; // it is wrong to use d here! NSLog(@"present age (seconds) %f", ti); // OS X 10.6 only NSDate *ageDoubled = [NSDate dateWithTimeInterval:ti sinceDate:[NSDate date]]; NSLog(@"age doubled %@", ageDoubled); NSDate *ageHalved = [NSDate dateWithTimeInterval:(ti/2) sinceDate:bd]; NSLog(@"age halved %@", ageHalved); return 0; }

NSTask

My initial attempts to include a C program file in a Cocoa-Python XCode project haven't worked. However, I do have a working example of using NSTask to run a C program as a separate process. I don't know if there is any advantage to doing it the Cocoa way (as opposed to using subprocess.Popen). The code in the AppDelegate is:

class PyXAppDelegate(NSObject): def applicationDidFinishLaunching_(self, sender): NSLog("Application did finish launching.") argL = ['13', '47', '36'] #p = NSHomeDirectory() + '/Desktop/example' p = NSBundle.mainBundle().pathForResource_ofType_('example', None) #print 'path', p t = NSTask.alloc().init() t.setLaunchPath_(p) t.setArguments_(argL) cent = NSNotificationCenter.defaultCenter() cent.addObserver_selector_name_object_( self, "checkATaskStatus:", NSTaskDidTerminateNotification, None) t.launch() @objc.signature('v@:@') def checkATaskStatus_(self, notification): status = notification.object().terminationStatus() if status == 0: print 'success' else: print 'error'

The C file looks (amost) like this:

#include < stdio.h> #include < stdlib.h> #include < string.h> #include < math.h> int isPrime(int x) { int i; float limit = sqrt(x) + 1; for (i = 2; i < limit; i++) { if (!(x % i)) return 0; } return 1; } int main(int argc, char *argv[]) { printf("number of args = %i \n", argc); printf ("path %s\n", argv[0]); int i; for (i = 1; i < argc; i++) { printf("arg #%i, ", i); int x = atoi(argv[i]); printf("x = %i, ", x); printf("isPrime = %i\n", isPrime(x)); } return 0; }

It's compiled in the usual way

tom-elliotts-mac-mini-2:~ telliott_admin$ gcc example.c -o example

and then can be placed in a convenient directory, or more flexibly, added to the project. Notable, pathForResource_ofType_ will take 'None' for the second argument.

This is what it prints to the Console:

2008-08-07 19:44:05.325 PyX[3207:10b] Application did finish launching. number of args = 4 path /Users/telliott_admin/Cocoa.PyObjC/PyX/build/Release/PyX.app/Contents/Resources/example arg #1, x = 13, isPrime = 1 arg #2, x = 47, isPrime = 1 arg #3, x = 36, isPrime = 0 success