I am reading a text, "Probability and Statistics" by Devore. I am looking at 2 items on page 740: the expected value and variance of the estimation of $\beta_1$, which is the slope parameter in the linear regression $Y_i = \beta_0 + \beta_1 X_i + \epsilon_i$. $\epsilon_i$ is a Gaussian($\mu = 0, variance=\sigma^2$) random variable and the $\epsilon_i$ are independent.
The estimate of $\beta_1$ can be expressed as: $\hat{\beta_1} = \frac{\sum (x_i – \bar{x}) (Y_i – \bar{Y})}{\sum(x_i-\bar{x})^2} = \frac{\sum (x_i – \bar{x})Y_i}{S_{xx}}$, where $S_{xx} = \sum (x_i – \bar{x})^2$. So, my question is: how do I derive $E(\hat{\beta_1})$ and $Var(\hat{\beta_1})$? The book has already given the results: $E(\hat{\beta_1}) = \beta_1$ and $Var(\hat{\beta_1}) = \frac{\sigma^2}{S_xx}$.
My work in the derivation: $E\left(\frac{\sum (x_i – \bar{x})Y_i}{S_{xx}}\right) = E\left(\frac{\sum (x_i – \bar{x})(\beta_0 + \beta_1 x_i + \epsilon)}{S_{xx}}\right) = E\left(\frac{\sum (x_i – \bar{x})\beta_1 x_i}{S_{xx}}\right)$, since $\sum(x_i – \bar{x})c = 0$ and $E(c\epsilon) = 0$. But I am stuck.
Also, $Var\left(\frac{\sum (x_i – \bar{x})Y_i}{S_{xx}}\right) = Var\left(\frac{\sum (x_i – \bar{x})(\beta_0 + \beta_1 x_i + \epsilon)}{S_{xx}}\right) = Var\left(\frac{\sum (x_i – \bar{x})\epsilon}{S_{xx}}\right) = Var\left(\frac{\sum (x_i – \bar{x})}{S_{xx}}\right) \sigma^2$, but I am stuck.
Best Answer
$E\left(\frac{\sum (x_i - \bar{x})\beta_1 x_i}{S_{xx}}\right)$ = $\frac{\sum (x_i - \bar{x})\beta_1 x_i}{S_{xx}}$ because everything is constant. The rest is just algebra. Evidently you need to show $\sum (x_i - \bar{x}) x_i = S_{xx}$. Looking at the definition of $S_{xx}$ and comparing the two sides leads one to suspect $\sum(x_i - \bar{x}) \bar{x} = 0$. This follows easily from the definition of $\bar{x}$.
$Var\left(\frac{\sum (x_i - \bar{x})\epsilon}{S_{xx}}\right)$ = $\sum \left[\frac{(x_i - \bar{x})^2}{S_{xx}^2}\sigma^2\right] $. It simplifies, using the definition of $S_{xx}$, to the desired result.