I know the standard way of finding the asymptotic distribution of the OLS estimator in a linear regression. Suppose
$$ y_i = x_i'\beta + u_i$$
where $u_i |X = x_i\sim N(0,\sigma^2)$.
Let $\hat{\beta}$ denote the OLS estimator of $\beta$. I know that
$$ \sqrt{n}(\hat{\beta} – \beta) \overset{d}\to N(0,\sigma^2 Q^{-1})$$
where $Q$ is the probability limit of $X'X$, as shown in page 17 of this document.
However, what's wrong with my attempt to derive the asymptotic distribution?
The CLT tells me
$$\sqrt{n} (\hat{\beta} – \beta) \overset{d}\to N(0, \mathrm{var}(\hat{\beta}))$$
Then define $U = [u_1, u_2, \dots, u_n]'$.
\begin{align}
\mathrm{var}(\hat{\beta}) &= E[(\hat{\beta} – \beta)^2 ] \\
&= E[( (X'X)^{-1}(X'U)) ( (X'X)^{-1}(X'U))'] \\
&= E[( (X'X)^{-1}(X'U)(U'X)(X'X)^{-1}] \\
&= E\bigg[(X'X)^{-1}\big(\sum_{i=1}^{n} x_i x_i' u_i^2 \big)(X'X)^{-1} \bigg] \\
&= E\bigg[(X'X)^{-1}\big(\sum_{i=1}^{n} x_i x_i' E[u_i^2|X] \big)(X'X)^{-1} \bigg] \\
&= E\bigg[(X'X)^{-1}\big(\sum_{i=1}^{n} x_i x_i'\big)(X'X)^{-1} \bigg]\sigma^2 \\
&= E\bigg[(X'X)^{-1}X'X(X'X)^{-1} \bigg]\sigma^2 \\
&= E[(X'X)^{-1}] \sigma^2
\end{align}
Obviously $E[(X'X)^{-1}] \neq Q^{-1}$, by Jensen's inequality. What went wrong?
Best Answer
That's because $$ \sqrt{n} (\hat{\beta}_n - \beta) \overset{d}{\not\to} N(0, \operatorname{Var}(\hat{\beta}_n)). $$ Instead, $$ \sqrt{n} (\hat{\beta}_n - \beta)\xrightarrow{d}N(0,\Sigma), $$ where $$ \Sigma=Q^{-1}\operatorname{Var}(x_1u_1)Q^{-1}=\sigma^2Q^{-1}QQ^{-1}=\sigma^2Q^{-1} $$ and $Q=\operatorname{plim}n^{-1}\sum_{i=1}^n x_ix_i^{\top}$.