I'm reading up on the Guass-Markov theorem on wikipedia, and I was hoping somebody could help me figure out the main point of the theorem.
We assume a linear model, in matrix form, is given by:
$$ y = X\beta +\eta $$
and we're looking for the BLUE, $ \widehat\beta $.
In accordance with this, I would label $\eta = y – X\beta$ the "residual" and
$\varepsilon = \widehat\beta – \beta$ the "error". (I.e the opposite of the usage on the Gauss-Markov page).
The OLS (ordinary least-squares) estimator may be derived as the argmin of $||\text{residual}||_2^2 = ||\eta||_2^2$.
Now, let $\mathbb{E}$ denote the expectation operator. To my understanding, what the Gauss-Markov theorem tells us is that, if $\mathbb{E}(\eta) = 0$ and $\text{Var}(\eta) = \sigma^2 I $, then the argmin, over all linear, unbiased estimators, of $\mathbb{E}(||\text{error}||_2^2) = \mathbb{E} (||\varepsilon||_2^2)$ is given by the same expression as the OLS estimator.
I.e.
$$ \text{argmin}_{\text{} \widehat\beta(y)} \, ||\eta||_2^2
\;=\; (X'X)^{-1}X'y \;=\;
\text{argmin}_{\text{linear, unbiased } \widehat\beta(y)} \, \mathbb{E}(||\varepsilon||_2^2)$$
Is my understanding correct? And if so, would you say that it deserves more prominent emphasis in the article?
Best Answer
I am not sure if I understood you question correctly, but if you are looking to prove that the OLS for $\hat{\beta}$ is BLUE (best linear unbiased estimator) you have to prove the following two things: First that $\hat{\beta}$ is unbiased and second that $Var(\hat{\beta})$ is the smallest among all linear unbiased estimators.
Proof that OLS estimator is unbiased can be found here http://economictheoryblog.com/2015/02/19/ols_estimator/
and proof that $Var(\hat{\beta})$ is the smallest among all linear unbiased estimators can be found here http://economictheoryblog.com/2015/02/26/markov_theorem/