Friday, July 10, 2009

Eigenvalues, part 4

Looking at the three matrices from the previous post, we can calculate the covariance matrix for each one:

cov(M)

           x           y
x 0.98197115 0.02222972
y 0.02222972 15.50833098


cov(M2)

        [,1]     [,2]
[1,] 8.26738 7.263180
[2,] 7.26318 8.222921


cov(M3)

             [,1]         [,2]
[1,] 1.550837e+01 4.074103e-16
[2,] 4.074103e-16 9.819371e-01


These are what we feed to eigen to calculate the eigenvalues and eigenvectors that tell us the magnitude and direction of maximum variance in the original matrices. (What I said at the end of this post was not correct, these are not scaled (i.e. not z-scores).

The values on the diagonal for cov(M) match (pretty closely) what we asked R to give us. The values in the x-vector were rnorm with sd = 1 and the y-vector had sd = 4. The variance is the square of this and matches what was observed (0.98 for x, 15.5 for y). The covariance is almost 0.

For M2, the vector rotated 45°, there is substantial covariance. And after we get the eigenvector for M2 and use it to rotate the M2 matrix to M3, now most of the variance is on the x-axis (1.550837e+01).