Solved – Derivation of OLS Variance

least squaresregressionself-studyvariance

I stumbled over a rather simple result of OLS regression which is

$$
Var[\hat\beta] = \sigma^2(X^TX)^{-1}
$$
where $\sigma$ is the variance of the error term $u$ and $X$ is the regressor matrix.

I first just accepted the proof in my textbook but now I am thinking that it either uses sloppy notation or I am missing something. $\hat\beta$ is the estimated and $\beta$ is the true parameter (assuming unbiasedness).

It states that

\begin{align}
Var[\hat\beta] &= E[(\hat\beta – \beta)(\hat\beta-\beta)^T] \\
&= E[(X^TX)^{-1}X^Tuu^TX(X^TX)^{-1}] \\
&= (X^TX)^{-1}X^T E[uu^T] X(X^TX)^{-1}
\end{align}

but $X$ was only assumed to be exogenous and not non-stochastic. Under this assumption I think $X$ cannot be dragged outside the expectation operator.

Momentarily, I think that it should be $Var[\hat\beta|X]$ to make sense. Is that the case? My web research couldn't clarify this. I only found similar derivations to the above without further explainations.

Best Answer

You are right that the conditional variance is not generally the same as the unconditional one. By the variance decomposition lemma, which says that, for r.v.s $X$ and $Y$

$$ Var(X)=E[Var(X|Y)]+Var[E(X|Y)] $$ Translated to our problem, $$ Var(\widehat{\beta})=E[Var(\widehat{\beta}|X)]+Var[E(\widehat{\beta}|X)] $$ Now, using that OLS is conditionally unbiased (under suitable assumptions like exogeneity assumed here), we have $$ E(\widehat{\beta}|X)=\beta $$ and thus $$ Var[E(\widehat{\beta}|X)]=0,$$ as $\beta$ is a constant, so that $$ Var(\widehat{\beta})=E[Var(\widehat{\beta}|X)] $$ or $$Var(\widehat{\beta})=E[\sigma^2(X'X)^{-1}]=\sigma^2E[(X'X)^{-1}].$$

Related Question