Let $u_1,\ldots,u_k$ be an orthonormal basis for the subspace $W \subset
\mathbb{R}^m$. Let $A = (u_1u_2\ldots u_k)$ be the $m \times k$ matrix whose
columns are the orthonormal basis vectors, and define $P=AA^T$ to be
the corresponding projection matrix.a.) Given $v \in \mathbb{R}^n$, prove that its orthogonal projection $w
\in W$ is given by matrix multiplication $w=Pv$
I am trying to understand what exactly is going on in this question. Here is an illustration I found that I wish you guys can help me understand. Correct me if I am wrong in any of my analysis of this image.
On this image the subspace $W\subset \mathbb{R}^m$ is the plane. The orthogonal basis $u_1,\ldots,u_k$ is the vectors $u_1, u_2$. Given $v\in \mathbb{R}^n$ I am not sure what represents that in the image, will it be $h^*_N$? For $w\in W$, will it be the orthogonal vector $f$ in the image?
My attempt:
I notice that $P =AA^T$ is equivalent to the least square solution. Also, will $v\in \mathbb{R}^n$ be orthogonal to every vector in the subspace $W$, so then $v = w – u$, where $u = c_1u_1,\ldots,c_ku_k$?
I will have to prove that w (which will equal the vector $f$ in the image) was given by $w=Pv$? Or is $v\in \mathbb{R}^n$ just $A^T$ since it is in $\mathbb{R}^n$ and not in $\mathbb{R}^m$ (which also I am having trouble understanding what exactly does $\mathbb{R}^n$ and $\mathbb{R}^m$ mean?).
Any thorough explanation will be greatly appreciated!
Best Answer
Note that $$\vec{w} = \sum_{r=1}^{k} \alpha_r \vec{u}_r = A\begin{bmatrix} \alpha_1, \alpha_2, \ldots, \alpha_k\end{bmatrix}^T = A \vec{\alpha}$$ Since $w$ is the orthogonal projection of $\vec{v}$, we have that $(\vec{v} - \vec{w}) \cdot \vec{u}_r = 0$ for all $\vec{u}_r$. Hence, we get that $$(v - w)^T u_r = 0\implies (v-w)^T A = 0 \implies A^T(v-w) = 0 \implies A^T v = A^T w = A^T A \alpha$$ Since the columns of $A$ are orthonormal, we have that $A^T A = I$. Hence, we get that $\alpha = A^Tv$. Hence, $$w=A \alpha = A A^T v$$