$X$ is a $T\times k$ matrix of constants with $\text{rank}(X)=k$, $Z$ is a $T\times q$ matrix of constants with $\text{rank}(Z)=q$, and $\text{rank}(X'Z)=k$. We have that
$$P_z = Z(Z'Z)^{-1} Z'$$
and
$$G= (X'P_zX)^{-1} – (X'X)^{-1}.$$
How can we prove that $G$ is positive semi-definite?
Best Answer
To prove that $G$ is psd is equivalent to proving that (see https://math.stackexchange.com/questions/435831/if-a-and-b-are-positive-definite-then-is-b-1-a-1-positive-semidef) $$ X'X- X'P_zX $$ is psd. Write this as $X'M_zX$ for the residual maker matrix $M_z=I-P_z$.
Now, let $d=Xc$ for some $c\neq0$. Then, $$ c'X'M_zXc=d'M_zd $$ By symmetry and idempotency of $M_z$ and letting $e=M_zd$, $$ d'M_zd=d'M_z'M_zd=e'e=\sum_ie_i^2, $$ which is a sum of squares and hence nonnegative.