I am trying to get an intuitive understanding of why there exists biased but consistent estimator.
Suppose $X_i \sim \mathcal{N}(\mu, 1)$. One example I came across is this:
$$W_n(X_1,\cdots,X_n) = \frac{1}{n} \sum_{i=1}^n X_i + \frac{1}{n}$$
I see that $W_n$ is biased due to the $\frac{1}{n}$ term, but is consistent. However, I feel that this example is somewhat "silly" because few people would use $W_n$ to estimate $\mu$.
Do we have any other example of biased but consistent estimator $W_n$, where people are more likely to use it?
Best Answer
Here's a straightforward one.
Consider a uniform population with unknown upper bound
$$ X \sim U(0, \theta) $$
A simple estimator of $\theta$ is the sample maximum
$$ \hat \theta = \max(x_1, x_2, \ldots, x_n) $$
This is a biased estimator. With a little math you can show that
$$ E[\hat \theta] = \frac{n}{n+1} \theta $$
Which is a little smaller than $\theta$ itself.
This also shows that the estimator is consistent, since $\frac{n}{n+1} \rightarrow 1$ as $n \rightarrow \infty$.
An natural unbiased estimator of the maximum is twice the sample mean. You can show that this unbiased estimator has much higher variance than the slightly biased on above.