Linear Regression – Proof that Trace of ‘Hat’ Matrix is Rank of X

linear algebralinear regressionmatricesstatistics

I understand that the trace of the projection matrix (also known as the "hat" matrix) X*Inv(X'X)*X' in linear regression is equal to the rank of X. How can we prove that from first principles, i.e. without simply asserting that the trace of a projection matrix always equals its rank?

I am aware of the post Proving: "The trace of an idempotent matrix equals the rank of the matrix", but need an integrated proof.

Best Answer

If $X$ is $n \times m$ with $m \le n$ and has full rank, then $rank (X) = \min(n,m) = m$, and we know $(X^T X)^{-1}$ exists.

By commutativity of the trace operator, we have

$$tr(H) := tr (X (X^T X)^{-1} X^T) = tr (X^T X (X^T X)^{-1} ) = tr[I_m] = m$$

Related Question