## Saturday, January 22, 2011

### Nullspace of a vector

This post is a note to myself about linear algebra.

Let's consider two vectors in R2:

 `a = ( 3, 4) = [ 3 ] [ 4 ]b = (-3, 1) = [ -3 ] [ 1 ]`

We can see from a plot that these vectors a and b are obviously not pointing in the same direction.

Also we notice that a is not a multiple (or a linear combination) of b (or vice versa). There is no constant such that when the elements of a are multiplied by the constant we get b.

Now consider a point P in R2, say

 `P = (2,-4)`

We can construct a linear combination of a and b that reaches this P or any other point.

How? Simply place one of the vectors at the origin and move along it (perhaps in a negative direction), and place the second vector at P and move along it, and find where the two lines meet.

Call the coordinates of the point where the lines cross x and y. There are actually two possibilities depending on which vector we choose for each role. Suppose we move in the reverse direction from the origin along a and from P along b. From the point-slope equation we know that:

 `(0 - y) / ( 0 - x) = 4/3 (from a)(Py - y) / (Px - x) = -1/3 (from b)(-4 - y) / ( 2 - x) = -1/3`

We can solve these two equations pretty easily. From the first

 `y = 4/3 x`

and from the second:

 `(-12 - 3y) = -(2 - x)x = -10 - 3y = -10 - 3(4/3) x5x = -10 x = -2 y = -8/3`

The other solution is symmetrical (we're dealing with a parallelogram). We would need to go +2 across and +8/3 up from P to reach x, y = (4, -4/3).

Suppose we want to know the actual multipliers for a and b, i.e. the fraction of the length of a and b that we travel along each vector.

From the figure, we can estimate that the values will be about -0.7 and -1.25.

Call these multipliers u and v. In matrix language

 `M = [ 3 -3 ] [ 4 1 ][ 3 -3 ] [ u ] = [ 2 ][ 4 1 ] [ v ] [ -4 ]`

One way we can solve the system (call it the Algebra 2 way) is as follows:

 `3u - 3v = 24u + v = -4`

Solve the second equation for v:

 `v = -4u - 4`

Plug into the first equation:

 `3u - 3(-4u - 4) = 215u = -10u = -2/3v = 8/3 - 4 = -4/3`

Finally, consider P not just as a point but as a vector c:

 `a = ( 3, 4)b = (-3, 1)c = ( 2,-4)`

Since we could have chosen P to be any point, c could be any vector in the x,y-plane and it would be constructed as a linear combination of a and b:

 `u a + v b`

We say that c is in the column space of the matrix whose columns are the vectors a and b:

 `M = [ 3 -3 ] [ 4 1 ]`

because we can get to c by taking a linear combination of the columns of M.

And of course, having a + b = c, we can also add -c to get:

 `u a + v b - c = 0`

If we think of another matrix as the column vectors a, b and c lined up:

 `[ 3 -3 2 ] [ u ] = [ 0 ][ 4 1 -4 ] [ v ] = [ 0 ] [ w ] = [ 0 ]`

The combination u = -2/3, v = -4/3, w = -1 solves this equation and brings us back to zero:

 `-2/3 [ 3 ] -4/3 [ -3 ] - [ 2 ] = [ 0 ] [ 4 ] [ 1 ] [ -4 ] [ 0 ][ -2 ] + [ 4 ] + [ -2 ] = [ 0 ][ -8/3 ] [ -4/3 ] [ 4 ] [ 0 ]`

That solution (a vector in R3):

 `x = [ u ] = [ -2/3 ] [ v ] [ -4/3 ] [ w ] [ -1 ]`

is said to be in the nullspace of

 `A = [ 3 -3 2 ] [ 4 1 -4 ]`

because Ax = 0.