Showing posts with label ssh. Show all posts
Showing posts with label ssh. Show all posts

Wednesday, August 31, 2011

Dissecting RSA keys in Python (4)

Still working with RSA keys. I accomplished the most important of the items left undone so far, to use the keys to encrypt a message using my own code.

In the process, I learned a new simple Python fact: the pow function (which is now a built-in), can take a modulus as a third argument.

And, it works where this doesn't seem to: (x**y) % z

(I discovered this by digging into the rsa module source).

python -m timeit -s "import rsa;  \
f = open('id_rsa');  data = f.read(); f.close(); k = rsa.PrivateKey.load_pkcs1(data)"\
 "41330915578951772302369**k.e % k.n"

10000 loops, best of 3: 79.4 usec per loop

python -m timeit -s "import rsa;  \
f = open('id_rsa');  data = f.read(); f.close(); k = rsa.PrivateKey.load_pkcs1(data)"\
 "pow(41330915578951772302369, k.e, k.n)"

10000 loops, best of 3: 70.2 usec per loop

In the example here, both work and have about the same timing. But in the code below, the first version hangs when doing decryption (with a large base). Here's the output, followed by the script.

> python encode.py 
m:   Hello, secret world!
p:   .xyz.Hello, secret world!.xyz.
a:   413309155789517723023698766343791993289928631329
c:   11434905702482726455415220687715293368262190253795 ..
i:   413309155789517723023698766343791993289928631329
r:   Hello, secret world!

encode.py

import rsa

with open('id_rsa') as f:
    data = f.read()
k = rsa.PrivateKey.load_pkcs1(data)
n = k.n
e = k.e
d = k.d

def my_atoi(s):
    L = [ord(c) for c in s]
    k = 256
    iL = L[:]
    iL.reverse()
    x = iL[0]
    for i in iL[1:]:
        x += i*k
        k *= 256
    return x

def my_itoa(i):
    rL = list()
    while i:
        rL.append(i%256)
        i = i/256
    rL.reverse()
    return ''.join([chr(n) for n in rL])

def encrypt(m):
    return m**e % n
    
def decrypt(c):
    # note:  c**d % n fails
    return pow(c,d,n)
    
if __name__ == '__main__':
    m = 'Hello, secret world!'
    pad = '.xyz.'
    p = pad + m + pad
    a = my_atoi(m)
    c = encrypt(a)
    i = decrypt(c)
    r = my_itoa(i)
    r = r.replace(pad,'')

    L = zip('mpacir',[m,p,a,c,i,r])
    N = 50
    for varname, var in L:
        s = str(var)
        print varname + ':  ', s[:N],
        if len(s) > N:  print '..'
        else:  print

Dissecting RSA keys in Python (3)

I'm working on understanding the structure of RSA keys as output by the ssh-keygen utility (previous posts here and here). At the risk of becoming redundant, I want to try to simplify what we did previously. Let's begin by rehashing old material on bytes in Python:

It's easy to get confused between a byte and its string representation, or for that matter, between an unsigned int and its string representation. Although hexadecimal is natural too, it's perhaps easiest to think of an individual byte in terms of the decimal equivalent (0..255). That's because a Python int converts easily to chr, bin, and hex.

>>> i = 35
>>> c = chr(i)
>>> c
'#'
>>> i == ord(c)
True
>>> b = bin(i)
>>> b
'0b100011'
>>> h = hex(i)
>>> h
'0x23'

Notice that hex and bin both give unpadded output:

>>> hex(1)
'0x1'
>>> bin(1)
'0b1'

Don't be fooled. h and b here are 'str' datatypes. But these string representations of bin and hex values go back to int easily (though we must specify the old base explicitly):

>>> int(b,2)
35
>>> int(h,16)
35

To interconvert hex and bin, it's easiest to go through int:

>>> bin(35)
'0b100011'
>>> hex(35)
'0x23'
>>> hex(int(bin(35),2))
'0x23'
>>> bin(int(hex(35),16))
'0b100011'

The struct module


This module performs conversions between Python values and C structs represented as Python strings. This can be used in handling binary data stored in files or from network connections, among other sources. It uses Format Strings as compact descriptions of the layout of the C structs and the intended conversion to/from Python values.


unpack works on the particular string representation of a byte in which it is a single character:

>>> i = 35
>>> c = chr(i)
>>> c
'#'
>>> unpack('B',c)
(35,)
>>> b = bin(i)
>>> unpack('B',b)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
struct.error: unpack requires a string argument of length 1
>>>

To construct a multi-byte value like a 4-byte unsigned int we can do this:

>>> h = '\x23'
>>> type(h)
<type 'str'>
>>> b_little = '\x23\x00\x00\x00'
>>> type(b_little)
<type 'str'>

With multi-byte objects (like ints) we get into endian-ness---'I' is the format for an unsigned int, and '>' means big-endian:

>>> unpack('I', b_little)
(35,)
>>> b_big = '\x00\x00\x00\x23'
>>> unpack('I', b_big)
(587202560,)
>>> unpack('>I', b_big)
(35,)

One can also use the binascii module, but this is more complicated. Let's write two bytes to a file and then read the binary data:

>>> fn = 'temp'
>>> FH = open(fn,'w')
>>> FH.write('##')
>>> FH.close()
>>> FH = open(fn,'rb')
>>> data = FH.read()
>>> FH.close()

This "data" is still a string:

>>> type(data)
<type 'str'>
>>> data
'##'
>>> len(data)
2
>>> b2a_hex(data[0])
'23'
>>> b2a_hex(data)
'2323'
>>> b2a_hex('#Az')
'23417a'

Or you can get fancy, but it seems unnecessary..

>>> import array
>>> L = array.array('B','#Az')
>>> type(L)
<type 'array.array'>
>>> L[0]
35
>>> L[1]
65
>>> L
array('B', [35, 65, 122])
>>> b2a_hex(L)
'23417a'

With all this in mind, we can redo the script from yesterday to read the base64-encoded key data in a way that is simply understandable. The output first:

> python read.py 
dlen 7
dlen 1
dlen 129

209,8,39,27 .. 169,97,145,127
0xd1,0x8,0x27,0x1b .. 0xa9,0x61,0x91,0x7f 

1467871546 .. 1722964351

We decode the data and break it into 3 parts exactly as before. Looking at the third part, we unpack it byte by byte. The result of unpacking is a list of ints. We convert that list directly into the large number n (by doing b * 256**i for each byte, moving along the list in reverse order), or we can convert the ints into hex values, assemble that list into one long hex string, and then call eval as before. It seems perfectly transparent now.

It even seems clear (in retrospect) why there is an additional null byte after the int that specifies the size of the third segment. It is probably to align the base64 encoding to begin with the first data byte.

The only remaining difficulty is that this approach fails for the private key. So that's still a mystery.

read.py
import base64
from struct import unpack

FH = open('data.txt','r')
data = FH.read()
FH.close()
b64_data = ''.join(data.strip().split())
data = base64.b64decode(b64_data)

L = list()
while data:
    dlen = unpack('>I',data[:4])[0]
    L.append(data[4:dlen+4])
    data = data[dlen+4:]
    print 'dlen', dlen
print

# let's look at n
bL = L[2]
bL = bL[1:]    # extra null first, why?
iL = [unpack('B',b)[0] for b in bL]
hL = [hex(i) for i in iL]

# look at the int and hex values
pL = [str(n) for n in iL]
print ','.join(pL[:4]), '..', ','.join(pL[-4:])
print ','.join(hL[:4]), '..',
print ','.join(hL[-4:]), '\n'

# first just do the computation ourselves
m = 256
iL.reverse()
n = iL[0]
for i in iL[1:]:
    n += i*m
    m *= 256
    
s = str(n)
print s[:10], '..', s[-10:]
    
# hex doesn't pad the output
# remove the '0x' and add the extra '0' if needed
for i in range(len(hL)):
    b = hL[i]
    if len(b) < 4:
        hL[i] = '0' + b[-1]
    else:
        hL[i] = b[-2:]  
h = '0x' + ''.join(hL)
nh = eval(h)
assert n == nh

Tuesday, August 30, 2011

Dissecting RSA keys in Python (2)

The public key from last time was given (in id_rsa.pub) as:

AAAAB3NzaC1yc2EAAAABIwAAAIEA0QgnG+LBQosx
pbkJIU0eWV9Zf7L63fy3BXp6T9RQH84yISs8SMvF
V4J1NYJQtNpGopNFlvzsuFnnxaRaM8ureFIQVa8k
/dRzv3r7QyIuoGyNQLDk3K8Tse/55fjuIxJFEf5j
voJs3Z/jYdPHreg8wzNCB/uMkbCccYsVt6lhkX8=

(It has been broken into 40 character lines). If we look at the first 32 characters of this base64 encoding:

AAAAB3NzaC1yc2EA
AAABIwAAAIEA0Qgn

the decoded version of this in hex is:

0x0  0x0  0x0 0x7  0x73 0x73 0x68 0x2d 0x72 0x73 0x61 0x0
0x0  0x0  0x1 0x23 0x0  0x0  0x0  0x81 0x0  0xd1 0x8  0x27

The first four bytes are

0x0 0x0 0x0 0x7

interpreted as a 32-bit unsigned int that is equal to 7, and indicates that the next 7 bytes are grouped together to form this element of the data (which is 'rsa-ssh', as I mentioned before).

0x73 0x73 0x68 0x2d 0x72 0x73 0x61
   s    s    h    -    r    s    a

The next 4 bytes after that are:

0x0 0x0 0x0 0x1

indicating the following block is a single byte of data: 0x23. In ASCII-encoding this would be the character '#', but it's actually an unsigned int:

>>> int('0x00000023', 16)
35

This is the value of the exponent for the key pair. More on this in a moment. Finally we get to

0x0 0x0 0x0 0x81

>>> int('0x00000081', 16)
129

It turns out there are exactly 129 bytes left in the data. In the code from the first post (from here), we used the struct module to unpack these three segments of data (we labeled them "parts").

According to the docs, the struct module can be used to interpret strings as packed binary data.


The first argument to unpack is a format string. The first character of the format string can be used to indicate the byte order, size and alignment of the packed data


The '>' indicates big-endian data, data laid out in registers in the same order as it is in memory. While Intel x86 (and current Macs) are little-endian, network order is big-endian and (I suppose) the SSH standard respects this. The next format character 'I' indicates the data following are unsigned ints.

import struct

hL = ['0x0','0x0','0x0','0x81']
iL = [int(h,16) for h in hL]
cdata = ''.join([chr(n) for n in iL])
res = struct.unpack('>I', cdata)

>>> print res
(129,)

It returns a tuple and we want the first value.

In the code from yesterday, we simply copied the correct number of bytes in each segment (as "characters"---not ASCII).

In the last part of this example, we need to transform those bytes into numerical data. We'll use struct.unpack again, except this time specifying binary data. For the exponent we had a single byte '\x23'', corresponding to character '#' which is 35 as an unsigned int:

>>> struct.unpack('B','#')
(35,)

Our function f passed the first element of this tuple back to a big list comprehension.

e = eval('0x' + ''.join(['%02X' % f(x) for x in parts[1]]))

In the case of the exponent (e) it operated on only a single value, so I'll rewrite it to clarify what this does. Remembering that f(x) will return 35, we'll get rid of the list stuff and just do:

>>> s = '%02X' % 35
>>> s
'23'
>>> e = eval('0x' + s)
>>> 
>>> e
35

(I'm not too swift with format strings, but you can read about them here).

And it doesn't do much in this case! But suppose we had two of these bytes in a row. Then:

>>> s = ('%02X' % 35) * 2
>>> s
'2323'
>>> eval('0x' + s)
8995
>>> 35*256 + 35
8995

eval will turn our string of bytes into a multi-byte unsigned int. This is news to me. So, if we actually had the string of bytes in hex divided into two sets of bytes, those up to and including '0x81'

['0x0', '0x0', '0x0', '0x7', '0x73', '0x73', '0x68', '0x2d', '0x72', '0x73', '0x61', '0x0', '0x0', '0x0', '0x1', '0x23', '0x0', '0x0', '0x0', '0x81']

and those after:

L = ['0x0', '0xd1', '0x8', '0x27', '0x1b', '0xe2', 
      '0xc1', '0x42', '0x8b', '0x31', '0xa5', '0xb9', '0x9', '0x21', '0x4d', '0x1e', '0x59', 
      '0x5f', '0x59', '0x7f', '0xb2', '0xfa', '0xdd', '0xfc', '0xb7', '0x5', '0x7a', '0x7a', 
      '0x4f', '0xd4', '0x50', '0x1f', '0xce', '0x32', '0x21', '0x2b', '0x3c', '0x48', '0xcb', 
      '0xc5', '0x57', '0x82', '0x75', '0x35', '0x82', '0x50', '0xb4', '0xda', '0x46', '0xa2', 
      '0x93', '0x45', '0x96', '0xfc', '0xec', '0xb8', '0x59', '0xe7', '0xc5', '0xa4', '0x5a', 
      '0x33', '0xcb', '0xab', '0x78', '0x52', '0x10', '0x55', '0xaf', '0x24', '0xfd', '0xd4', 
      '0x73', '0xbf', '0x7a', '0xfb', '0x43', '0x22', '0x2e', '0xa0', '0x6c', '0x8d', '0x40', 
      '0xb0', '0xe4', '0xdc', '0xaf', '0x13', '0xb1', '0xef', '0xf9', '0xe5', '0xf8', '0xee', 
      '0x23', '0x12', '0x45', '0x11', '0xfe', '0x63', '0xbe', '0x82', '0x6c', '0xdd', '0x9f', 
      '0xe3', '0x61', '0xd3', '0xc7', '0xad', '0xe8', '0x3c', '0xc3', '0x33', '0x42', '0x7', 
      '0xfb', '0x8c', '0x91', '0xb0', '0x9c', '0x71', '0x8b', '0x15', '0xb7', '0xa9', '0x61', 
      '0x91', '0x7f', '0x0']

Note: The extra byte at the end here is caused by our having decoded by hand.

Paste it into the interpreter. Then, they have to be padded out...

for i in range(len(L)):
    s = L[i]
    c = s[-1]
    if len(s) < 4:  
         L[i] = '0' + c
    else:  
        L[i] = s[-2:]
Remove the first and last bytes and turn it into one long hex value:
>>> s = '0x' + ''.join(L[1:-1])
>>> eval(s)
146787154640181728129549360865206451974125178122992752499327844555082827713683060312609720092642289502088305950292885968241446393671268243539585900329449529436170858131743078225065287541399805133121852823856839448386268013284889428740476278604926908637093742329548018673369795976229837442944484597101722964351L

That matches what we got yesterday, and what the rsa module gave us for n from the private key.

This use of eval seems to be widely known, but I would never have appreciated it from just reading the docs.

Here is a quick script that shows all the values for each set of four characters in the base64 encoding, first 9 bytes of output first:

> python script2.py 
['A', 'A', 'A', 'A']
[0, 0, 0, 0]
['0b0', '0b0', '0b0', '0b0']
['000000', '000000', '000000', '000000']
['00000000', '00000000', '00000000']
0x0 0x0 0x0
0 0 0 

['B', '3', 'N', 'z']
[1, 55, 13, 51]
['0b1', '0b110111', '0b1101', '0b110011']
['000001', '110111', '001101', '110011']
['00000111', '01110011', '01110011']
0x7 0x73 0x73
7 115 115 

['a', 'C', '1', 'y']
[26, 2, 53, 50]
['0b11010', '0b10', '0b110101', '0b110010']
['011010', '000010', '110101', '110010']
['01101000', '00101101', '01110010']
0x68 0x2d 0x72
104 45 114

from string import *
v = True

cL = list(uppercase + lowercase + digits + '+/')
D = dict(zip(cL,range(len(cL))))
D['='] = 0

key = '''AAAAB3NzaC1yc2EAAAABIwAAAIEA0QgnG+LBQosxpbkJIU0eWV9Zf7L63fy3BXp6T9RQH84yISs8SMvFV4J1NYJQtNpGopNFlvzsuFnnxaRaM8ureFIQVa8k/dRzv3r7QyIuoGyNQLDk3K8Tse/55fjuIxJFEf5jvoJs3Z/jYdPHreg8wzNCB/uMkbCccYsVt6lhkX8='''

L = list(key)

while L:
    s = L[:4]
    L = L[4:]
    if v:  print s
    dL = [D[c] for c in s]
    if v:  print dL
    dL = [bin(c) for c in dL]
    if v:  print dL
    bL = list()
    for b in dL:
        n = 8 - len(b)
        b = n*'0' + b[2:]
        bL.append(b)
    if v:  print bL
    b = ''.join(bL)
    bL = [b[:8], b[8:16], b[16:]]
    if v:  print bL
    iL = [int('0b' + b,2) for b in bL]
    for i in iL:
        print hex(i),
    print
    for i in iL:
        print i,
    print '\n'

Monday, August 29, 2011

Dissecting RSA keys in Python

I want to look at RSA keys a little bit more in this post. Let's generate a public/private pair (1024 bits) using the SSH utility keygen (no passphrase):

> ssh-keygen -b 1024 -t rsa 
Generating public/private rsa key pair.
Enter file in which to save the key (/Users/telliott_admin/.ssh/id_rsa): id_rsa
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in id_rsa.
Your public key has been saved in id_rsa.pub.
The key fingerprint is:
96:49:5d:40:b1:90:de:99:c5:05:43:f2:92:45:69:67 telliott_admin@c-98-236-78-154.hsd1.wv.comcast.net
The key's randomart image is:
+--[ RSA 1024]----+
|        .o=*B+.  |
|        .o B=.E  |
|       ...==.o   |
|       ..o+.     |
|        S        |
|       .         |
|                 |
|                 |
|                 |
+-----------------+

[ To be honest, at this point, I'm not actually sure this is the randomart that goes with the example in this post.. but I think it could be :) ]

Start with the "public" key. The data in the file looks like this:

id_rsa.pub
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEA0QgnG+LBQosxpbkJIU0eWV9Zf7L63fy3BXp6T9RQH84yISs8SMvFV4J1NYJQtNpGopNFlvzsuFnnxaRaM8ureFIQVa8k/dRzv3r7QyIuoGyNQLDk3K8Tse/55fjuIxJFEf5jvoJs3Z/jYdPHreg8wzNCB/uMkbCccYsVt6lhkX8= my_identifier

Where my_identifier is

telliott_admin@c-myipaddress_with_dashes.hsd1.wv.comcast.net

The common representation of RSA keys (like we see in the file above) uses base64 format. I was confused about this at first, but found a great explanation here, and of course in wikipedia:


The base64, base32, and base16 encodings convert 8 bit bytes to values with 6, 5, or 4 bits of useful data per byte, allowing non-ASCII bytes to be encoded as ASCII characters for transmission over protocols that require plain ASCII, such as SMTP. The base values correspond to the length of the alphabet used in each encoding. There are also URL-safe variations of the original encodings that use slightly different results.


Consider the byte 1001-1110

We can encode this byte using hexadecimal representation as '9e'

>>> b = 0b10011110
>>> b
158
>>> hex(b)
'0x9e'

I didn't actually know you could do this, i.e. using 0b with no quotes (see SO).

The encoding of those 8 bits (one byte) as ASCII characters takes 2 bytes. The '9' and the 'e' take up that much space. An advantage of this representation of the byte is that it's easy to read, also the information can be transmitted to devices that are expecting an ASCII string, where the highest-value bit should be unset.

A disadvantage is that it doubles the size of the string that we're transmitting. As the quote indicates, the other encodings cram more bits of information into the byte to be transmitted. In base32 we could use (RFC4648) the characters 'A'..'Z' plus '2'..'7'.

In base64 we use uppercase, lowercase, digits plus '+' and '/', except that:


Using standard Base64 in URL requires encoding of '+', '/' and '=' characters into special percent-encoded hexadecimal sequences ('+' = '%2B', '/' = '%2F' and '=' = '%3D'), which makes the string unnecessarily longer.
For this reason, a modified Base64 for URL variant exists, where no padding '=' will be used, and the '+' and '/' characters of standard Base64 are respectively replaced by '-' and '_',


The wikipedia article has a great graphic showing an encoding in detail. We do conversions easily by using the Python base64 module.


>>> import base64
>>> e = base64.b64encode
>>> d = base64.b64decode
>>> e('Man')
'TWFu'
>>> d('TWFu')
'Man'
>>> s = 'Fourscore, and seven years ago'
>>> len(s)
30
>>> c = e(s)
>>> c
'Rm91cnNjb3JlLCBhbmQgc2V2ZW4geWVhcnMgYWdv'
>>> d(c)
'Fourscore, and seven years ago'


The encoding process looks at each sequence of 24 bits in the input (3 bytes) and encodes those same 24 bits spread over 4 bytes in the output. The last two characters [in the next example], the ==, are padding because the number of bits in the original string was not evenly divisible by 24 in this example.


>>> e('Ma')
'TWE='

We break this two byte (16 bit) value up into 6 + 6 + 6 = 18 bits---that makes three base64 values, and the third one is now E rather than F because the bit pattern is 000100, where the last two 00 are padding. Additional adding is added to bring us back to the byte boundary.

In Lincoln's famous phrase, there are 30 bytes, which is evenly divisible by 6, hence no padding is needed.

The second part of the problem was solved by a question I found on SO

Here is my version of the code:

import base64
import struct

data = open('id_rsa.pub').read().split(None)[1]
print data[:24]
keydata = base64.b64decode(data)
print keydata[:18]

parts = []
while keydata:
    dlen = struct.unpack('>I',keydata[:4])[0]
    data, keydata = keydata[4:dlen+4], keydata[4+dlen:]
    parts.append(data)

def f(x):
    return struct.unpack('B', x)[0]
    
e = eval('0x' + ''.join(['%02X' % f(x) for x in parts[1]]))
print 'e', e

n = eval('0x' + ''.join(['%02X' % f(x) for x in parts[2]]))
print 'n', n

I have to confess I haven't sorted this out completely yet. Nevertheless, it allows us to convert the RSA public key from ssh-keygen to an integer:

Output:

> python script.py
AAAAB3NzaC1yc2EAAAABIwAA
ssh-rsa#
e 35
n 146787154640181728129549360865206451974125178122992752499327844555082827713683060312609720092642289502088305950292885968241446393671268243539585900329449529436170858131743078225065287541399805133121852823856839448386268013284889428740476278604926908637093742329548018673369795976229837442944484597101722964351

I want to finish by showing that these values are actually correct, and tie the whole thing back to an earlier discussion of the mathematics of public key cryptography.

I found a module for Python that handles RSA cryptography.

One very nice thing about it is that it's pure Python. It might not be fast, but there's nothing to do to build it. (I used easy_install). It turns out that the rsa module can read the private RSA key that we got earlier (top of the post), but not the public key (which is why I learned everything that's come before this point):

> python
Python 2.7.1 (r271:86832, Jun 16 2011, 16:59:05) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import rsa
>>> with open('id_rsa') as f:  data = f.read()
... 
>>> data.split()[4][:50]
'MIICWgIBAAKBgQDRCCcb4sFCizGluQkhTR5ZX1l/svrd/LcFen'
>>>
>>> k = rsa.PrivateKey.load_pkcs1(data)
>>> k.n
146787154640181728129549360865206451974125178122992752499327844555082827713683060312609720092642289502088305950292885968241446393671268243539585900329449529436170858131743078225065287541399805133121852823856839448386268013284889428740476278604926908637093742329548018673369795976229837442944484597101722964351L
>>> k.e
35
>>> k.d
71296617968088267948638260991671705244575086516882194071102095926754516318074629294696149759283397758157177175856544613145845391211758861147798865874304045384903321200809536161346504709634159813307741235395231601606901597492591281205810328850837168587984349087962868939575631849929827946396907884195208774355L
>>> k.p
12214944519618901215070692893221534580187337690484280451921546594832793966376942655508370083135703324858458340368238021755353370324360760471666368429071147L
>>> k.q
12017013618393528112233953119917677248859562342149056793052943111068006176296374093622600767249370271694572460567324457033873003984957249202580919628769533L
>>> k.n == k.p * k.q
True
>>>

You can see that n equals p * q and that it matches what we eventually obtained from the public key file (id_rsa.pub) above. That's pretty cool.

Still to do: use the keys to do encryption and decryption and check that the math is right. Figure out how the hash/digest is done. And figure out how the passphrase is used to modify the private key file. We can even add a passphrase at this point to the old version:

ssh-keygen -f id_rsa -p

I used 'abcde' as a passphrase (weak!):

> cat id_rsa
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-128-CBC,FB4D3BD371510C4B37552EF6E949CC48

E+KtbKVAwE4Dxj3vKYgfHpbgqF9Nk6O4vJ+C6wogXEHXr8vkM9SF5pz6lKNfGRbC
S93/lurvq+1+rCuldDNmEjdy36cbgrWJ1CIwB6fq5VyT3+v+AxDMqV9nQkvHjnxR
/ON2n+CjOSklnvmaBk3l8+IvCIKar2hGS/1qHqJRKJ3B1fgd5I8i9yH6hx8yl8Qm
AfVPgBJ4+91dCW2Y8odtXENgjstGtWZkYd0O+EqPuSuPHLu+J7M2KSQNe8aKOhGf
DXCEwR4aGW8EXfRNQpULbi5JruYebYlg3JILm2MaFwy+OW39q1VElvBc5wD01awW
BF7Y9x++bi9krX/G49NzkLdSVshkDMSP+eRZ4sncWvDCrkN2xlWqnNfQ06I+XCVW
FMUmyMFI87eNmjcZOgIi1S+fQtWcbl4o9Hx1z6xcMd7WxOnfIxe5zHIu8C24WONW
spcVxSsHAA57jYa8j292prtK6HDDa6UOAa32hBz1lvC8fMWPxwluTJ8eyeeK/7O6
hNI5G9gM4MsMEMau8O49fEm0RuiT/aB8IXZeEVQNbV/FdixyBBhY53DMnFQ5wau8
OJCpL51S5lj2o8eAJc61SHwX/Fc9uF+rmAFy08tthLuCF2wGdZWsE7IJ8NQzFJA+
Us2dTCjY57V/QoCC41rWh+YP+srcEK1Wp7DfvXcq98N1/+tECyCe/aDdPCIM/8QD
glKrx8Ah7Tsx6of3vCHiYG0q8Yu68NNJOiWrnGMEmP0Gq9yhLYqKrc8a3oIP0Lst
Ss+0p989df73GdM3nyqtLUwAzaCJQnDivLFT6gltEnM=
-----END RSA PRIVATE KEY-----

Sunday, August 28, 2011

Linux Server SSH

To continue with my Ubuntu Linux server odyssey, I got SSH working this afternoon for access from my LAN. As always, heed the warnings, these posts are without a warranty of any kind, not least because I am a total amateur. They exist because it's easy to have Google to "index my brain" or rather, what was in my brain at a certain time.

The objective is to enable remote login for my Ubuntu machine running with a local ip address of 10.0.1.2. We begin by setting a stronger password for my account on Ubuntu under: System > Preferences > About Me (weirdly, it's not under Passwords and Encryption Keys ..).

On the client (OS X Snow Leopard Server), I already have some key pairs from a previous test

> ls ~/.ssh
authorized_keys	id_dsa		id_dsa.pub	known_hosts


but they are DSA keys, which the notes I'm going to follow deprecate. We'll make a short 1024 byte bit RSA pair for now:

ssh-keygen -b 1024 -t rsa

> cat id_rsa.pub 
ssh-rsa AAAAB..5Eoec= telliott_@c-98-___-__-154.hsd1.wv.comcast.net

Not sure what the pseudo ip address or "comcast" thing is about.

I also take this opportunity to set a passphrase for the public key (and of course I make a note of it). This will be useful (it says) because we can set up the server to require both a password to gain access to the public key, as well as the corresponding private key before login is allowed.

[UPDATE: I'm still working my way through this, but I suspect that the previous statement isn't correct. The passphrase is used to protect the value of the private key. So, in this setup OS X will decrypt the private key using the passphrase, and the stored value of the private key in id_rsa is encrypted. I'll try to figure all this out soon. ]

The easiest way to get my new public key onto the server is to use SSH (with password, before we disable it). The next question is, do we need to install an SSH server? It appears yes.

sudo apt-get install openssh-server

There is a pre-existing config file /etc/ssh/ssh_config but after the previous command there are more including sshd_config. I make sure to save a copy of this file before I modify it. Now try:

ssh te@10.0.1.2

ssh: connect to host 10.0.1.2 port 22: Connection refused

OK, so we need to modify /etc/ssh/sshd_config. Port22 is already uncommented. Now uncomment:

PermitRootLogin no
ChallengeResponseAuthentication yes
PasswordAuthentication yes   # we'll set it to no eventually

On the server:

sudo /etc/init.d/ssh restart

On the client:

> ssh te@10.0.1.2
The authenticity of host '10.0.1.2 (10.0.1.2)' can't be established.
RSA key fingerprint is d1: .. :83.
Are you sure you want to continue connecting (yes/no)? 

Check the fingerprint on the the server:

ssh-keygen -l -f /etc/ssh/ssh_host_rsa_key.pub

2048 d1: .. :83 /etc/ssh/ssh_host_rsa_key.pub (RSA)

Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.0.1.2' (RSA) to the list of known hosts.
Connection closed by 10.0.1.2

> ssh te@10.0.1.2
te@10.0.1.2's password: 
Welcome to Ubuntu 11.04 (GNU/Linux 2.6.38-11-generic x86_64)

Now to change to not using our password. We need to copy our RSA public key to the server in some secure way. According to the sshd_config file, the "authorized keys file" is
%h/.ssh/authorized_keys. That's in my home directory. I do:

> scp ~/.ssh/id_rsa.pub te@10.0.1.2:~/.ssh/authorized_keys
te@10.0.1.2's password: 
id_rsa.pub                                    100%  260     0.3KB/s   00:00    

te@VB:~$ logout
Connection to 10.0.1.2 closed.

> ssh te@10.0.1.2
Identity added: /Users/telliott_admin/.ssh/id_rsa (/Users/telliott_admin/.ssh/id_rsa)
Welcome to Ubuntu 11.04 (GNU/Linux 2.6.38-11-generic x86_64)

* Documentation:  https://help.ubuntu.com/

Last login: Sun Aug 28 14:35:49 2011 from osxserver.local
te@VB:~$ 

Now, finally, be sure to turn off password authentication for the ssh server: PasswordAuthentication no. At first, I'm not being prompted for a password to retrieve my public key on the server.. just getting automatically logged in by:

ssh te@10.0.1.2

I forgot to do:

ChallengeResponsePasswords yes

sudo /etc/init.d/ssh restart

Restart and I get the challenge.. Although it is apparently cached sometimes. Looks like it works.

Saturday, February 5, 2011

Secure communications (3)

This is the third post in a series about remote login using ssh under OS X. The first two posts are here and here. This one is working through the details of the server/client setup using RSA, as described in this O'Reilly article (here, printable). The article also has a link to a page for online testing (of your security setup) by Symantec.

I don't normally bother to remind you that (as it says in the profile) YMMV---your mileage may vary. If you follow the notes here that I describe what I did, and your computer is reduced to a smoking heap of twisted metal in under 10 seconds, well, all I can do is to repeat myself: YMMV. So be careful.

For the server, we first need to set up a directory for keyfiles:


mkdir ~/.ssh


We also need to turn on remote login:


System Prefs > Sharing > Remote login  


I always have everything locked down (nothing selected under Sharing) but for today we'll turn on the lights and the stereo. In the middle of the System Prefs window it will say something like "to log in to this computer remotely type":


ssh username@ipaddress


where ipaddress is something like 10.0.X.X. The default SSH port is port 22, which will be unblocked through the firewall automatically after you do this. Now, if you are on the same subnet as the server, from a second machine (say, my laptop) you can do:


ssh username@ipaddress

The authenticity of host '10.0.X.X (10.0.X.X)' can't be established.
RSA key fingerprint is [ 32 hexadecimal digits ]


This is not the key itself, but its fingerprint, which is much more readable and sufficient for verifying that it is the correct value.

Continues:


Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '10.0.X.X' (RSA) to the list of known hosts.
Password:
Last login: Sat Feb 5 15:19:19 2011
>


At the point where I typed yes, we were about to log in remotely. However, we could be fooled by a malicious evesdropper. At that point, we need to go to the server and do:


ssh-keygen -l -f /private/etc/ssh_host_rsa_key.pub


This will echo the public key for the server, and you should confirm that it matches what you got from ssh above. And if you look on the client (in a second Terminal window) in ~/.ssh:


> cat ~/.ssh/known_hosts 
10.0.X.X ssh-rsa __ [ the same server public RSA key ]


To quote the article:

You normally should not see this message again since SSH has written down the key it asked you to check as a "trusted" key for future reference. If you see it again this may mean that someone is impersonating your server. You should immediately disconnect your computer, pick up your phone, and call your network administrator.


From the logged in client enter:


> say I sure love being inside this fancy computer


and hear the message on the server. The default is not Victoria any longer (for that add the option -v Victoria). Our log in currently uses password authentication, but we should really use something more secure. On the client do:


> ssh-keygen -b 1024 -t dsa
Generating public/private dsa key pair.
Enter file in which to save the key (/Users/telliott/.ssh/id_dsa):


Hit return to choose the default. It will prompt for a password---use a strong password. Then, you will get something like this:


Your identification has been saved in /Users/telliott/.ssh/id_dsa.
Your public key has been saved in /Users/telliott/.ssh/id_dsa.pub.
The key fingerprint is:
[ 32 hexadecimal digits ]

telliott@localhost.local
The key's randomart image is:


I won't show you the image, but it's pretty cool. Having generated a DSA (cousin of RSA) key pair, we copy our public key over to the server using a secure copy protocol in the current session (which was started with the log in password):


scp ~/.ssh/id_dsa.pub username@serverip:~/.ssh/authorized_keys

The server wants a password (the account log in password) and outputs:

Password:
id_dsa.pub 100% 620 0.6KB/s 00:00
>


Test login by quitting Terminal and restarting, then do:


> ssh username@ipaddress
Identity added: /Users/telliott/.ssh/id_dsa (/Users/telliott/.ssh/id_dsa)
Last login: Sat Feb 5 15:27:51 2011 from 10.0.X.X
>


It works! At this point it would be a good idea to disable automatic login on startup for your client laptop. If a black hat has access to this machine, (s)he could also easily compromise (at least) your account on the server.

Another thing is that if you fail to authenticate by the key (above) the SSH server will fall back on password authentication. We need to edit a file on the server to disable that service. In fact, I recommend turning off Remote Login until we figure that out.

Secure communications (2)

The wikipedia article has a worked example of RSA. I'm going to shamelessly appropriate it for the following discussion.


Choose p = 61 and q = 53
Compute n = pq = 3233
Compute φ(n) = (p-1)(q-1) = 60 times 52 = 3120
Choose a number 1 < e < 3120 that is coprime to 3120


Coprime (or relatively prime) means that e and 3120 should have no common (positive) factor other than 1. Let's try e = 17. Since e is prime, we just have to check that e does not divide 3120 exactly.


>>> 17 * 183 
3111
>>> 17 * 184
3128
>>> 3120 % 17
9


e is called the public key exponent.

So the public key would be (n = 3233, e = 17). The encryption function is


m**17 (mod 3233)


The plaintext message would be padded first with extra bits. Now, suppose we want to encrypt the message m = 65. In Python we do m**e % n:


>>> (65**17) % 3233
2790L


Our ciphertext is 2790.

We compute the private key exponent, d, as the modular multiplicative inverse of e (mod φ(n)).

Consider an integer a. The modular multiplicative inverse of a is designated a-1 where


a a-1 = 1 (mod m)


Here then, we want d such that d * e = 1 (mod 3120). There is an algorithm called the extended Euclidean algorithm to do this. Here we simply note that d = 2753 is a solution since:


>>> (2753 * 17) % 3120
1


Note that the calculation required (effective) knowledge of p and q through the use of φ(n).

In order to decrypt the ciphertext 2790, we do c**d % n:


m = (c**2753) % 3233

(2790**2753) % 3233
65L


That is some serious exponentiation.

From the terminal I do:


> ssh-keygen


The file has this:


ssh-rsa AAAAB3NzaC1yc2EAAAAB..


I'm very surpised to see that this is like ascii alphanumeric or something. I'll have to look into that.

[ UPDATE: it is base 64 (RSA format here), which I'd never heard of but makes a lot of sense: uppercase + lowercase + digits = 62. We just need two more symbols, which are '+' and '/' (here). ]

A key size of 2048 bits would be common for RSA. This would give about a 600 digit number in base 10:


>>> len(str(2**2048))
617


Not sure what part of that is e versus n, but most of it is probably n, and since pq = n we're talking about quite large prime numbers.

Secure communications (1)

I've been interested in cryptography and the whole question of secure communication over an insecure network, but never had the time to get into it.

I thought we could take a crack at it here. :)

The place to start is public key cryptography. I found a nice (though old) series at O'Reilly on ssh, which includes a very basic description of the cryptography part in the middle of the first article (link / printable). It's written to show Mac users how to use ssh (sorry). There is also a nice and much more extensive article at wikipedia.

As described in the wikipedia article on RSA (an algorithm for public key encryption), there are the following steps:


- choose two distinct large prime numbers p and q about the same size
- compute n = pq
- compute φ(n) = (p-1)(q-1)
- choose e, the public key exponent, e.g. 0x10001 = 65537
- the private key exponent, d, is computed from e and φ(n)
- the public key is then (n,e)
- the private key is d


The computation to obtain d is described here.

Naturally, the private key is kept secret, while the public key is published.

The important thing is that the encryption and decryption methods have the property that decryption with the private key is the inverse of encryption with the public key:

ciphertext = public key encryption (plaintext)
plaintext = private key decryption (ciphertext)


Even better, the converse is also true:

ciphertext = private key encryption (plaintext)
plaintext = public key decryption (ciphertext)


Thus, Alice can use Bob's public key to encrypt a message that only Bob can decrypt. Bob's problem is that anyone can do this and claim to be Alice, since the encryption used only his public key.

Authentication, or signing, of messages relies on hash functions (like md5). The value computed by md5 from a message (sometimes called a fingerprint) is sensitive to small changes in the message, and also it is "impossible" to fashion a message to match a given hash.

Alice then adopts a modified strategy:


Alice encrypts a message for Bob using his public key
Alice uses md5 to compute a hash of the message
Alice encrypts the hash using her private key
Bob reverses the operations


Only Bob can read the message. He authenticates Alice as the sender because he can decode the hash using her public key and see that it matches what md5 gives him from the message.

The version I've seen of this protocol uses the encrypted version of the message to generate the hash. But I'm not really sure whether you do md5 on the plaintext of the message or the encrypted version, and can't see that it makes a difference as far as Bob is concerned. However, I've also read that an attacker having both plaintext and ciphertext for some message is a weakness, so I think it's better if the hash is of the plaintext for the original message, to protect Alice's private key.

Eve (an evesdropper) cannot read the message, lacking Bob's private key. She cannot substitute a different message of her own, because the hash won't match. She cannot substitute both a different message and a hash, because then when Bob decrypts the hash using Alice's public key, it doesn't match the hash from Eve's message.

There are two more issues to think about right away. The first is that this process is really expensive computationally. So usually, public key cryptography is used to exchange a secret key that is then used for a block or stream cipher.

The second is the question of the validity of public keys. That's the whole purpose of digitally signed certificates (wikipedia).

If you look in the Keychain under Certificates > System Roots, you'll see a (frighteningly long---163 items for me) list of Certificate Authorities whose public key we trust implicitly, and who in turn guarantee say, Bank of America, when you get a logon page for online banking.

But, I'm not so interested in banking. I'm interested in ssh and virtual private networks, and Kerberos.. all sorts of things.

If you know something about this subject (actually the subject of any post) and see an error, or have something to add, feel free to comment.