The question is related to Semiparametric Theory Ch3 by Tsiatis
First consider a Hodge estimator:
Let $Z_1, \ldots, Z_n$ be iid $N(\mu, 1), \mu \in \mathbb{R}$. For this simple model, we know that the maximum likelihood estimator (MLE) of $\mu$ is given by the sample mean $\bar{Z}_n=n^{-1} \sum_{i=1}^n Z_i$ and that
$$
n^{1 / 2}\left(\bar{Z}_n-\mu\right) \stackrel{\mathcal{D}(\mu)}{\longrightarrow} N(0,1) .
$$
Now, consider the estimator $\hat{\mu}_n$ given by Hodges in 1951 (see LeCam, 1953):
$$
\hat{\mu}_n=\left\{\begin{array}{lll}
\bar{Z}_n & \text { if } & \left|\bar{Z}_n\right|>n^{-1 / 4} \\
0 & \text { if } & \left|\bar{Z}_n\right| \leq n^{-1 / 4}
\end{array}\right.
$$
Next here is the definition of regular estimator:
Consider a local data generating process (LDGP), where, for each $n$, the data are distributed according to " $\theta_n$," where $n^{1 / 2}\left(\theta_n-\theta^*\right)$ converges to a constant (i.e., $\theta_n$ is close to some fixed parameter $\theta^*$ ). That is,
$$
Z_{1 n}, Z_{2 n}, \ldots, Z_{n n} \quad \text { are } \quad \text { iid } p\left(z, \theta_n\right)
$$
where
$$
\theta_n=\left(\beta_n^T, \eta_n^T\right)^T, \quad \theta^*=\left(\beta^{*^T}, \eta^{*^T}\right)^T .
$$
An estimator $\hat{\beta}_n$, more specifically $\hat{\beta}_n\left(Z_{1 n}, \ldots, Z_{n n}\right)$, is said to be regular if, for each $\theta^*, n^{1 / 2}\left(\hat{\beta}_n-\beta_n\right)$ has a limiting distribution that does not depend on the LDGP.
Meanwhile, the author showed an example of what happened to the Hodges estimator if the true estimator $\mu_n = n^{-1/3}$ – one can derive $n^{1 / 2}\left(\hat{\mu}_n-\mu_n\right) \text { diverges to }-\infty \text {. }$
To that end, the author claimed that the constructed Hodges estimator is not regular.
However, I don't think the condition meet with the one in the definition. In particular,
$n^{1 / 2}\left(\mu_n-0\right) = n^{1/6} \to \infty$, instead of a constant. My question is – must the definition of regular estimator go with the rate $n^{1/2}$?
Best Answer
I don't think the book is implying that $\mu_n=n^{-1/3}$ is the right choice to show that Hodges estimator is not regular, it is merely saying that it is easy to check that Hodges estimator is not regular. Admittedly it is not very clearly written. We can consider $\mu_n = t/\sqrt{n}$ for a fixed $t$ and $\mu^* = 0$. Then $\sqrt{n}(\mu_n - \mu^*) = t$, and it is straight forward to argue that the law of $\sqrt{n} (\hat{\mu}_n - \mu_n)$ is going to depend on $t$. In other words, the limiting distribution depends on the specific choice of sequence $\mu_n$. This is the key point of regularity. An estimator that is regular will have limiting distribution that does not depend on the data generating distribution (e.g. it will be independent of $t$), it is not crucial to have the dependence on the $\sqrt{n}$-rate, though this is common since $\sqrt{n}$ plays a key role in statistics (parametric rate), see the discussion here for more on this point.
Here is a different example that is also nice to keep in mind and is taken from the book "Efficient and Adaptive Estimation for Semiparametric Models":
Suppose that $X_1,\dots,X_n$ are i.i.d. $N(\theta, I_k)$ in $\mathbb{R}^k$ for $k\ge 3$. Consider the James-Stein estimator of the mean ($\theta$) defined as:
$$ \hat{\theta}_n = \left ( 1-\frac{k-2}{n \| \overline{X}\|^2_2}\right )\overline{X} $$
Now suppose that $\theta^*=0$, and take $\theta_n= t/\sqrt{n}$ for a fixed $t \in \mathbb{R}^k$. Now, introduce the notation $\mathcal{L}_{\theta_n}$ to denote the "law/distribution under the measure $\mathbb{P}_{\theta_n}$", then
$$ \mathcal{L}_{\theta_n} (\sqrt{n}(\overline{X} - \theta_n))= \mathcal{L}_{0}(X_1) = N(0,I_k), $$ then under $\mathbb{P}_{\theta_n}$, for any $n \ge 1$,
\begin{align*} \sqrt{n}(\hat{\theta}_n - \theta_n) &= \sqrt{n}(\overline{X} - \theta_n) - \frac{k-2}{\|\sqrt{n}(\overline{X} - \theta_n) + t\|_2^2}(\sqrt{n}(\overline{X} - \theta_n) + t)\\ &=_d X_1 - \frac{k-2}{\| X_1+t\|^2_2}(X_1+t), \end{align*} in words, the limiting distribution depends on $t$, so $\hat{\theta}_n$ is not regular at $\theta^* = 0$.