I am trying to derive (or prove) the relationship between the eigenvalues and eigenvectors of the matrices $X'X$ and $XX'$. It is fairly intuitive that they are related but I cannot derive the relationship. The result is simply stated in passing as part of another proof in a multivariate stats methods book but when I tried to work it out I couldn't. The $X$ are sample vectors so real numbers, say dimension ($n \times p$). The result is:
If $l_k$ and $\mathbb a_k$ are the $k^{th}$ eigenvalue and eigenvector of $X'X$, then the $k^{th}$ eigenvalue and eigenvector of XX' are $l_k$ and ${l_k}^{-1/2}X\mathbb a_k$.
Can someone point me to a good reference explaining this or show me how?
Many thanks for the help.
Best Answer
The underlying result is more general:
If $\lambda \neq 0$ is an eigenvalue of $AB$ with eigenvector $v$, then $ABv = \lambda v$. Hence $BABv = BA (Bv) = \lambda Bv$ (and $Bv \neq 0$, otherwise $ABv=0$).
Hence if $\lambda \neq 0$ is an eigenvalue of $AB$ with eigenvector $v$, then $\lambda$ is an eigenvalue of $BA$ with eigenvector $Bv$.
Take $A=X^T$, $B=X$.
Note that if $X=\begin{bmatrix} 1 \\ 0 \end{bmatrix}$, then $X^T X=1$ has exactly one eigenvalue at $1$, but $X X^T $ has eigenvalues $\{0,1\}$.