Solved – Consistent estimator, that is not MSE consistent

biasconsistencyestimationmseunbiased-estimator

I'm trying to understand the concept of consistency in point estimation. Could somebody give me an illustrative example of a (simply) consistent estimator that is not MSE consistent?

Just a recall of definitions and theorems (I hope I have them right):

  • $\hat{\theta}_m$ is simply consistent estimator of parameter $\theta$ if $\forall \epsilon > 0$: $$\lim_{m \to \infty} \mathbb{P}[| \hat{\theta}_m – \theta | > \epsilon] = 0$$

  • $\hat{\theta}_m$ is MSE consistent estimator of parameter $\theta$ if: $$\lim_{m \to \infty} \mathbb{E}[(\hat{\theta}_m – \theta)^2] = 0$$

  • MSE can be decomposed into variance and bias:
    $$ \mathbb{E}[(\hat{\theta} – \theta)^2] = \underbrace{\mathbb{E}[(\hat{\theta} – \mathbb{E}[\hat{\theta}])^2]}_{\text{Var}[\hat{\theta}]} + \underbrace{(\mathbb{E}[\hat{\theta}] – \theta)^2}_{\text{Bias}_\theta[\hat{\theta}]^2} $$

  • MSE consistency implies simple consistency from Chebyshev's Inequality:
    $$\mathbb{P}[|\hat{\theta} – \theta| \geq \epsilon] \leq \frac{\mathbb{E}[(\hat{\theta} – \theta)^2]}{\epsilon^2}$$

Best Answer

Suppose $X_1, X_2, \cdots X_n \stackrel{iid}{\sim} N(\mu, 1)$ and our goal is to estimate $\mu$. Consider the estimator $$\hat\mu_n = \begin{cases} \bar X_n, & \text{with probability $\frac{n-1}{n}$} \\[1.2ex] n, & \text{with probability $\frac{1}{n}$} \end{cases}$$

By introducing $Y_n \sim Bern\left(\frac{n-1}{n}\right)$ with $Y_i \perp \!\!\! \perp Y_j$ (for all $i\neq j$) and $Y_i \perp \!\!\! \perp X_j$ (for all $i$ and $j$) we can rewrite this estimator as $$\hat\mu_n = \bar X_n Y_n + n(1-Y_n) .$$

This estimator is simply consistent.

To see this, consider \begin{align*} P(|\hat\mu_n - \mu| < \epsilon) &= P(|\bar X_n - \mu| < \epsilon)\frac{n-1}{n} + P(|n-\mu|<\epsilon)\frac{1}{n} \\[1.3ex] &=\frac{n-1}{n}\left\{\Phi\left(\sqrt n \epsilon \right)- \Phi\left(-\sqrt n \epsilon \right) \right\} + P(|n-\mu|<\epsilon)\frac{1}{n} \end{align*}

Taking the limit gives $$\lim_{n\rightarrow\infty} P(|\hat\mu_n - \mu| < \epsilon) = 1,$$ and thus $\hat\mu_n \stackrel{p}{\rightarrow} \mu$.

This estimator is not MSE-consistent

We start by finding the bias.

\begin{align*} E(\hat\mu_n) &= E(\bar X_n Y_n + n(1-Y_n)) \\[1.2ex] &= E(\bar X_n)E(Y_n) + nE(1-Y_n) \\[1.2ex] &= \mu\frac{n-1}{n} + n\frac{n-1}{n} \\[1.2ex] &= \frac{n-1}{n}\mu + 1 \end{align*}

Therefore $B_\mu(\hat\mu_n) = 1 - \mu/n$ which does not converge to $0$ as $n \rightarrow \infty$. Since $Var(\hat\mu_n)$ must be non-negative this is enough to conclude that $MSE(\hat\mu_n) \not\rightarrow 0$, and therefore the estimator is not MSE-consistent.

Related Question