Suppose that I have a $n \times k$ real matrix with full column rank. Say $k=3$ and I write
$$X = [\mathbf x_1:\mathbf x_2:\mathbf x_3]$$
where lower-case $\mathbf x$'s are $n \times 1$ vectors.
I go on and form the orthogonal projection matrix
$$P_X = X \left(X'X\right)^{-1}X'$$.
Consider now the matrix
$$W = [\mathbf x_1:-\mathbf x_2:\mathbf x_3]$$
Namely it is equal to $X$ matrix, but in (any) one column, the sign of the elements are switched.
Question: Can we express the projection matrix of $W$, $P_W=W \left(W'W\right)^{-1}W'$, in terms of the projection matrix of $X$, $P_X$, or at least state some relation between them?
I tried to explore this with what little matrix algebra I know, but could not come up with anything. In reality the $k$ dimension is bigger that $3$ but I guess this does not matter.
Best Answer
We have $P_X = P_W$.
We can show this using matrix algebra by noting that $W = XQ$, where $Q$ is the orthogonal matrix $$ Q = \pmatrix{1&0&0\\0&-1&0\\0&0&1}. $$ With that, we note that $$ \begin{align} P_W &= W(W'W)^{-1}W' = [XQ]([XQ]'[XQ])^{-1}[XQ]' \\ & = XQ[Q' (X'X) Q]^{-1}Q'X \\ & = XQ[Q'(X'X)^{-1}Q]Q'X \\ & = X[QQ'](X'X)^{-1}[QQ']X = X(X'X)^{-1}X = P_X, \end{align} $$ as was desired.