I am trying to derive the finite sample variance of an OLS estimator when the regressor is also random. More concretely, I am looking at the following case:
$$
Y_i = \beta X_i + \epsilon_i
$$
where
$$
X_i \sim \mathcal{N}(0, \sigma^2_x) \\
\epsilon_i \sim \mathcal{N}(0, \sigma^2_\epsilon)
$$
and $X_i$ and $\epsilon_i$ are independent. I know that the OLS estimator $\widehat{\beta}$ is:
$$
\widehat{\beta} = \frac{\sum X_i Y_i}{\sum X^2_i}
$$
The $X_i$ and $\epsilon_i$ are i.i.d.
I want to compute the finite sample variance of $\widehat{\beta}$. I have only come across variance results which assume that the regressor $X$ is fixed, i.e., for $\text{Var}(\widehat{\beta}|X)$.
Note, I was able to derive the asymptotic variance of $\widehat{\beta}$. I am stuck on the finite sample case.
Thanks for your help!
Best Answer
\begin{align} Var(\hat{\beta}) &= E(Var(\hat{\beta}|X)) + Var(E(\hat{\beta}|X))\\ &= E\left( \frac{\sigma^2_{\epsilon}}{\sum X_i^2} \right) + Var(\beta)\\ \end{align} As $X_i$ are i.i.d normal random variables with zero mean and variance of $\sigma^2_{x}$, thus $\sum X_i^2/\sigma_x^2$ is distributed $\chi^2(n)$ hence $(\sum X_i^2/\sigma_x^2)^{-1}$ is distributed Inverse chi squared with with mean of $1/(n-2)$, thus \begin{align} Var(\hat{\beta}) &= E\left( \frac{\sigma^2_{\epsilon}}{\sigma_x^2} \text{Inv-}\chi^2(n) = \right)=\frac{\sigma^2_{\epsilon}}{(n-2)\sigma_x^2} \end{align}