Consider an ordinary least squares model,
$$y = \beta X + \epsilon \qquad \epsilon\sim N(0, \sigma)$$
The Gauss-Markov theorem tells us that the ordinary least-squares (OLS) estimator is the minimum-variance linear unbiased estimator (BLUE) for the coefficients:
$$ \beta \approx \hat\beta = (X^TX)^{-1}X^Ty $$
Does an unbiased, nonlinear estimator with lower variance, $\tilde\beta$, exist?
Based on my previous question.
Best Answer
The Gauss-Markov theorem gives the conditions where the OLS estimator is the BLUE, and those conditions do not include normality of the residuals. When we also include that normality assumption, then we can remove the "L" and wind up with the "Best Unbiased Estimator", not just the best linear unbiased estimator (section 2.1, example 1 of the Ohio State econometrics notes).
However, if we do not make the normality assumption, then we can wind up with nonlinear estimators of the coefficients that have lower variance than the OLS estimate but are unbiased. For example, consider heavy-tailed errors and the solution given by minimizing absolute loss (quantile regression at the median), as I do here.