Do robust estimators like M-estimator still have higher variance than OLS in presence of non-normal errors and/or outliers

estimatorsleast squareslinearnormality-assumptionunbiased-estimator

In my studies I've learned that even with non-normality of the errors, the OLS estimator is still considered BLUE (Best Linear Unbiased Estimator). The texts also suggested using M and L estimators for robust estimation in those situations.

But if M-estimators like Huber are still linear (correct me if im wrong) estimators, why do we use them?

One reason being suggested often is that even if robust estimators are not "best", they're less sensitive to violations of assumptions or outliers than OLS. However, doesn't being "less sensitive to.." means they have lower variance in those situations? Then how could OLS still be the "best" there?

Best Answer

M- and other robust estimators are not linear in the observations. Note that this is a different issue from fitting a linear model; it concerns how the estimator is computed! In fact being linear in the observations means that outliers cannot be down-weighted, which is what robust estimators aim to do.

In fact they do have (sometimes dramatically) lower variance under heavy tailed error distributions and outlier-generating models (you can simulate this if you want to convince yourself), however they are not better under all non-normal distributions. OLS can be better under distributions with lighter tails than the normal.

Note by the way that if you want to be robust against leverage points, you need to do something more sophisticated than a standard M-estimator, for example an MM- or tau-estimator (see robustbase package in R).