Solved – How to derive the standard error of linear regression coefficient

inferencestandard error

For this univariate linear regression model
$$y_i = \beta_0 + \beta_1x_i+\epsilon_i$$
given data set $D=\{(x_1,y_1),…,(x_n,y_n)\}$, the coefficient estimates are
$$\hat\beta_1=\frac{\sum_ix_iy_i-n\bar x\bar y}{n\bar x^2-\sum_ix_i^2}$$ $$\hat\beta_0=\bar y – \hat\beta_1\bar x$$
Here is my question, according to the book and Wikipedia, the standard error of $\hat\beta_1$ is $$s_{\hat\beta_1}=\sqrt{\frac{\sum_i\hat\epsilon_i^2}{(n-2)\sum_i(x_i-\bar x)^2}}$$
How and why?

Best Answer

3rd comment above: I've already understand how it comes. But still a question: in my post, the standard error has (n−2), where according to your answer, it doesn't, why?

In my post, it is found that $$ \widehat{\text{se}}(\hat{b}) = \sqrt{\frac{n \hat{\sigma}^2}{n\sum x_i^2 - (\sum x_i)^2}}. $$ The denominator can be written as $$ n \sum_i (x_i - \bar{x})^2 $$ Thus, $$ \widehat{\text{se}}(\hat{b}) = \sqrt{\frac{\hat{\sigma}^2}{\sum_i (x_i - \bar{x})^2}} $$

With $$ \hat{\sigma}^2 = \frac{1}{n-2} \sum_i \hat{\epsilon}_i^2 $$ i.e. the Mean Square Error (MSE) in the ANOVA table, we end up with your expression for $\widehat{\text{se}}(\hat{b})$. The $n-2$ term accounts for the loss of 2 degrees of freedom in the estimation of the intercept and the slope.

Related Question