Solved – Proving $E(\hat{\sigma}^2)=\sigma^2$

regressionself-studyunbiased-estimatorvariance

I'm trying to prove $E(\hat{\sigma}^2)=\sigma^2$, where $\hat{\sigma}^2$ is the error variance and $\sigma^2$ the population variance.

I'm working on the following equation:

$\sum \hat{u}_i^2=\sum(u_i-\bar{u})^2+(\hat{\beta_1}-\beta_1)^2\sum(x_i-\bar{x})^2-2(\hat{\beta}_1-\beta_1)\sum u_i(x_i-\bar{x})$

I'm having trouble proving that $\sum(u_i-\bar{u})^2=(n-1)\sigma^2$

Can someone give me a hint on how to start?

$u_i$ is the error term and $\bar{u}$ is the average of the error term, not the expected value.

proof:

$\sum (u_i-\bar{u})^2=E[\sum u_i^2 – n(\bar{u})^2]$

$\sum E(u_i^2)-E(n(\bar{u})^2)$

$n\sigma^2-nE(\bar{u})^2$

$n\sigma^2-nE(\bar{u})^2$

$n\sigma^2-nE(\frac{\sum u_i}{n})^2$

$n\sigma^2-\frac{1}{n}E(\sum u_i)^2$

$n\sigma^2-\frac{1}{n}E(\sum u_i \sum u_i)$

Best Answer

Little wonder you're having trouble proving $\sum(u_i-\bar{u})^2=(n-1)\sigma^2$, since it isn't the case.

What you should be proving is $E\left[\sum(u_i-\bar{u})^2\right]=(n-1)\sigma^2$.

There are some complete derivations to be found on site - including in the regression context and even using the same notation as you have. Some can be found in the related links in the sidebar to the right.

Outside the regression context (which isn't necessary to the derivation) there are a number of other derivations on site, some relatively simple, some a bit more complicated.

To get you started -- the most obvious strategy is to expand the square and take the expectation inside the sum, then simplify. You may need to rely on some known basic facts about variance and expectation.

Edit:

Thanks for showing some working.

  1. The first line is wrong. You need to be careful about leaving out $E()$s -- make sure they don't just appear and disappear at random times.

  2. Be careful of how you write the expectation of a squared term; in particular if you mean $E[(\bar{u})^2]$ you should probably write that because if you write $E(\bar{u})^2$ it might be interpreted as $(E[\bar{u}])^2$.

  3. You can write the square of the sum of $u_i$ out term by term and see that it's the sum of the squares plus twice the sum of cross products $u_iu_j$, (for $i<j$). However...

  4. If you use basic properties of variance to write down the variance of $\bar{u}$ you'll probably find it easier to find $E[(\bar{u})^2]$

Related Question