Least Squares – Proving OLS Unbiasedness Without Conditional Zero Error Expectation

least squaresunbiased-estimator

The OLS estimate $b$ is equal to $(X^TX)^{-1}X^Ty$ for the linear regression model. If we assume that $E(\epsilon|X)=0$ then it is easy to prove simply by taking the conditional expectation, of $b$ substituting in the expression for $y$ and simplifying.

But how do we prove it if we only know that $E(X^T\epsilon)=0$? $E(\epsilon|X)=0$ implies $E(X^T\epsilon)=0$, but not the other way around.

EDIT: Can I just get an answer, just to be sure, that even if the $u_i$'s are i.i.d., that $E(x_iu_i)=0$ does not imply unbiasedness?

So just to be absolutely clear: If $u_i$ is i.i.d., and we know that $E(x_iu_i)=0$ holds, but we don't know whether $E(u_i|x_i)=0$ holds, then OLS may be biased?

Best Answer

You can't, because the statement is not true under the weaker assumption.

Consider for example the autoregressive model \begin{equation*} y_{t}=\beta y_{t-1}+\epsilon _{t}, \end{equation*} in which the strict exogeneity $E(\epsilon|X)$ is violated even under the assumption $E(\epsilon_{t}y_{t-1})=0$:

we have that \begin{equation*} E(\epsilon_ty_{t})=E(\epsilon_t(\beta y_{t-1}+\epsilon _{t}))=E(\epsilon_{t}^{2})\neq 0. \end{equation*} But, as $y_{t+1}=\beta y_{t}+\epsilon_{t+1}$, $y_t$ is also a regressor for $y_{t+1}$ and hence, it is impossible in this model that the error term is also uncorrelated with future regressors.

Now, it is also well-known that OLS is biased for the coefficient of an AR(1)-model, see Why is OLS estimator of AR(1) coefficient biased?

Related Question