[Math] Computing the matrix that represents orthogonal projection,

Question

There is a theorem that says if $U$ is an orthogonal matrix, i.e., its columns (or rows) form an orthonormal basis, then the action of $UU^T$ represents orthogonal projection of the vector space onto the space spanned by the columns of $U$.

But, why does this theorem also apply when my "$U$" matrix is a $3\times1$ column vector and hence not an orthogonal matrix (it's not even square)?

I had to compute the matrix that represents orthogonal projection onto the line spanned by $(1, 2, -1)$. So, following the Gram-Schmidt process for this simple case, just set $v_1= (1,2,-1)$. Then I normalized this vector to make it of unit length. Finally, I compute $VV^T$ to get a $3\times3$ matrix. I applied the theorem stated above and concluded that this $3\times3$ matrix represents orthogonal projection onto the line spanned by the one column in $V$, which has the same span as the span of $(1, 2,-1)$, since the Gram-Schmidt process produces a set of orthogonal vectors with the same span as the original set of linearly independent vectors.

Have I applied the theorem incorrectly? I got the correct matrix.

Thanks,

Answer 1

The theorem you have quoted is true but only tells part of the story. An improved version is as follows.

Let $U$ be a real $m\times n$ matrix with orthonormal columns, that is, its columns form an orthonormal basis of some subspace $W$ of ${\Bbb R}^m$. Then $UU^T$ is the matrix of the projection of ${\Bbb R}^m$ onto $W$.

Comments

• The restriction to real matrices is not actually necessary, any scalar field will do, and any vector space, just so long as you know what "orthonormal" means in that vector space.
• A matrix with orthonormal columns is an orthogonal matrix if it is square. I think this is the situation you are envisaging in your question. But in this case the result is trivial because $W$ is equal to ${\Bbb R}^m$, and $UU^T=I$, and the projection transformation is simply $P({\bf x})={\bf x}$.