Solved – Are linear regression errors independent? Mean independent? Uncorrelated

assumptionserrorlinear modelproofregression

All I know is that we assume zero conditional mean (and hence zero mean) and conditional homoscedasticity (and hence homoscedasticity).

When trying to prove that $E[(\hat{\beta_1} – \beta_1)\bar{u}] = 0$, where $\beta_1$ is the slope in the linear regression model, $\hat{\beta_1}$ is its estimate and $\bar{u}$ is the average of the errors in the linear regression model (not the residuals!), I encountered:

$$E[(\hat{\beta_1} – \beta_1)\bar{u}|x]$$

$$\vdots$$

$$ = \frac{1}{n}\sum_{i=1}^{n} \frac{(x_i – \bar{x})}{SST_x} \color{red}{[\sum_{j=1}^{n} E[(u_i)u_j|x]]}$$

$$ = \frac{1}{n}\sum_{i=1}^{n} \frac{(x_i – \bar{x})}{SST_x} \color{red}{\sigma^2}$$

$$\vdots$$

$$ = 0 $$

$$\to E[(\hat{\beta_1} – \beta_1)\bar{u}] = 0$$

QED


What is the justification for that part? I tried:

For $i \ne j$, we have $E[(u_i)u_j|x] = Cov[u_i,u_j|x] + E[(u_i)|x]E[u_j|x] \stackrel{(*)}{=} 0 + (0)(0) = 0$

For $i = j$, we have $E[(u_i)u_j|x] = E[(u_i^2)|x] = Var[u_i|x] = \sigma^2$

Is $(*)$ right?

If so, what is the justification?

If not, how does one show that $E[u_i u_j | x] = 0$?


From Wooldridge:


enter image description here


This is from $(ii)$ of this exercise:


enter image description here

Best Answer

The key thing here is that Wooldridge makes the assumption of random sampling.

Notice that since we have a random sample this means $(x_i, y_i) \perp (x_j, y_j)$ for $i \neq j$ which means that the single components of the pair are also independent, in particular $x_i \perp x_j$ (to see that just notice the joint is $p(x_i, y_i, x_j, y_j) = p(x_i, y_i)p (x_j, y_j)$ and marginalize over $y$).

This further implies $y_i \perp y_j |x_i, x_j$, since:

$$ p(y_i, y_j|x_i, x_j) = \frac{p(x_i, y_i, x_j, y_j)}{p(x_i, x_j)} = \frac{p(x_i, y_i)p (x_j, y_j)}{p(x_i)p(x_j)} = p(y_i|x_i)p(y_j|x_j) $$

But $y_i|x_i$ is nothing more than the disturbance $u_i$ plus a constant. Hence $u_i \perp u_j |x$ when assuming a random sample and $E[u_i u_j | x] =E[u_i|x] E[u_j | x] = 0$