Let $(x_i, Y_i)\in\mathbb{R}^2$ be independent observations on $n$ subjects, such that $$Y_i|x_i\sim N(x_i\beta, \sigma^2)$$ where $(\beta, \sigma^2)\in\mathbb{R}^2$ are unknown coefficients.
I computed the maximum likelihood estimate $\hat\beta$ of $\beta$, which is $\hat\beta = \frac{\sum_{i=1}^n y_{i}x_i}{\sum_{i=1}^n x_i^2}$, and we want to compute the variance of this estimator $\hat\beta$.
Using that $Var(\hat\beta)= E[\hat\beta^2]-E[\hat\beta]^2$, I would only need $E[\hat\beta^2]$ to get the variance, as I already showed $E[\hat\beta]=\beta$, but I'm struggling with it.
$$E[\hat\beta^2]=E[(\frac{\sum_{i=1}^n y_{i}x_i}{\sum_{i=1}^n x_i^2})^2]=\frac{1}{(\sum_{i=1}^n x_i^2)^2}E[(\sum_{i=1}^n y_{i}x_i)^2]$$
I do not really know how to compute this expectation. Any help would be appreaciated.
Best Answer
$y_{i}|x_{i} \sim \mathcal{N}(x_{i}\beta | \sigma^2) \Leftrightarrow y_{i} = \beta x_{i} + \epsilon_{i}, \ \epsilon_{i} \sim \mathcal{N}(0, \sigma^2)$
It's easier to find $\mathbb{D}(\hat \beta)$ directly - not by finding $\mathbb{E}(\hat{\beta}^2)$.
$\mathbb{D}(\hat \beta) = \mathbb{D}\left( \frac{\sum\limits_{i=1}^n y_{i}x_{i}}{\sum\limits_{i=1}^n x_{i}^2} \right) = \frac{1}{\left( \sum\limits_{i=1}^n x_{i}^2 \right)^2} \mathbb{D}\left( \sum\limits_{i=1}^n (x_{i}\beta + \epsilon_{i})\cdot x_{i} \right) = \frac{1}{\left( \sum\limits_{i=1}^n x_{i}^2 \right)^2} \mathbb{D}\left( \sum\limits_{i=1}^n \epsilon_{i}\cdot x_{i} + \beta \cdot \underbrace{\sum\limits_{i=1}^n x_{i}^2 }_{\text{nonrandom}} \right) = \frac{1}{\left( \sum\limits_{i=1}^n x_{i}^2 \right)^2} \mathbb{D}\left( \sum\limits_{i=1}^n \epsilon_{i}\cdot x_{i} \right) = \frac{1}{\left( \sum\limits_{i=1}^n x_{i}^2 \right)^2} \sum\limits_{i=1}^n \mathbb{D}(\epsilon_{i})\cdot x_{i}^2 = \frac{\sigma^2 \cdot \sum\limits_{i=1}^n x_{i}^2 }{\left(\sum\limits_{i=1}^n x_{i}^2 \right)^2} = \boxed{\frac{\sigma^2}{\sum\limits_{i=1}^n x_{i}^2}}$