I want to find the following example: Construct a linear model $y=X\beta + \epsilon$, where $\epsilon $ is mean zero uncorrelated homoscedestic non-Gaussian noise and $X$ is deterministic, such that there exists a nonlinear unbiased estimator that has a strictly smaller variance than the least square estimator.
I have no idea how to proceed.
Best Answer
I would start with the one-sample case. Given the trivial linear model $y=\mu+\epsilon$, where $\epsilon$ is mean zero uncorrelated homoscedestic noise, the OLS estimator is just the sample average. What are some non-linear estimators of $\mu$? Can you choose $\epsilon$ so one of them is better than the sample average? If so, can you expand this to where $\mu=X\beta$ is non-trivial.
One choice for a non-linear estimator would be the median, so you need to find a distribution for $\epsilon$ where the median is unbiased for the mean but has strictly smaller variance than the mean.