Let $w_1,…,w_n$ be any basis of the subspace $W \subset
\mathbb{R^m}$. Let $A = (w_1,…,w_n)$ be the $m$ x $n$ matrix whose
columns are the basis vectors, so that $W = rngA$ and $rankA=n$. Let
$P = A(A^TA)^{-1}A^T$ be the corresponding projection matrix.a.) Prove that the orthogonal projection of $v \in \mathbb{R^n}$ onto
$w \in W$ is obtained by multiplying by the projection matrix: $w=Pv$.b.) Show that if $A=QR$, then $P = QQ^T$. Why is $P \ne I$?
How will I be able to prove these?
Best Answer
For part b), note that $A^T A = R^T Q^T Q R = R^T R$, so \begin{align*} P &= QR(R^T R)^{-1} R^T Q^T \\ &= Q R R^{-1} R^{-T} R^T Q^T \\ &= Q Q^T. \end{align*}
For part a), suppose $b \in \mathbb{R}^m$, and let $\hat{b} = A \hat{x}$ be the projection of $b$ onto $W$. The residual $b - A \hat{x}$ is orthogonal to $W$, hence it is orthogonal to each of the columns of $A$. This tells us that \begin{align*} &A^T (b - A \hat{x}) = 0 \\ \implies & A^T A \hat{x} = A^T b \\ \implies& \hat{x} = (A^T A)^{-1} A^T b \\ \implies& A \hat{x} = A (A^T A)^{-1} A^T b \\ \implies& \hat{b} = A (A^T A)^{-1} A^T b. \end{align*}
This shows that the matrix $P = A (A^T A)^{-1} A^T$ projects onto $W$.