[Math] Techniques for proving asymptotic normality by Taylor expansion

probabilityprobability theorystatistical-inferencestatistics

Suppose I have a sequence of densities

$$
f_{X_n}(x) = \exp[\ell_n(x)], \qquad (x \in A).
$$

My goal is to prove a statement like $\sqrt n (X_n – \mu) \to N(0, \sigma^2)$ in distribution, for an appropriate $\sigma^2$ and some choice of $\mu$. We often teach undergraduates to do this using moment generating functions or characteristic functions, or by direct manipulation of the cdf.

By contrast, I would like to do this by expanding $f_{X_n}(x)$ about the mode $\hat x_n$ (suppose $f_{X_n}(x)$ is unimodal with $\ell''_n(x) < 0)$. Then we expand $T_n = \sqrt n (X_n – \hat x_n)$

$$
f_{T_n}(t) \approx n^{-1/2}f_{X_n}(\hat x_n) \exp \left\{\frac 1 {2n} t^2 \ell''_n(\hat x_n)\right\}.
$$

Under the condition that $\hat x_n = \mu + o(n^{-1/2})$ and $\ell''_n(\mu) / n \to -I(\mu)$ for some function $I(\mu)$, one then might guess

$$
T_n \stackrel{\cdot}{\sim} N(0, I(\mu)^{-1}).
$$

Now for the question: I'm essentially looking for conditions which allow me to make this result rigorous. The heuristic seems very similar to what one does when trying to prove asymptotic normality of maximum likelihood estimators, and so I would hope for some conditions in terms of $\ell''_n(x)$ and $I(x)$, or perhaps a third derivative $\ell'''_n(x)$.

The best I can do at the moment is try to expand things out
$$
f_{T_n}(t) = n^{-1/2} \exp\left[\ell_n (\hat x_n) + \underbrace{t / \sqrt n \ell_n(\hat x_n)}_{=0} + \frac{t^2}{2n}\ell''_n(\hat x_n + n^{-1/2} t^\star)\right]
$$
with remainder $\frac 1 {2n} t^2 \ell''(\hat x_n + n^{-1/2} t^\star)$, which works for showing that the unnormalized density converges to a Gaussian density. But this doesn't finish the job because I still have to show that the normalizing constant converges to the right answer if I want to finish off the proof with Scheffe's theorem. I could try to finish it by applying dominated convergence to the exponential term, but this hasn't worked so far for the problem I'm interested in, and doesn't feel like a sensible thing to do anyways.

Best Answer

See "Testing statistical hypotheses" by E.L. Lehmann (196), Chap. 6 regarding regularity conditions. If you don't have access to that, here is a link to them. See p. 6-12 of the pdf.

To summarize, you need to demonstrate a bit more than just consistency of the mode. Specifically, you need to show:

  1. $\lim_{n \to\infty} E(X_n)=\lim_{n \to\infty} \int xf_n(x) dx = \mu < \infty$
  2. $\lim_{n \to\infty} Var(X_n) = I(\mu)^{-1}\in[0,\infty)$
  3. The quadratic approximation needs to be unfiormly $O_p\left(\frac{1}{\sqrt{n}}\right)$ in a neighborhood around $\mu$.
  4. $\mu \notin \{\inf A,\sup A\}$

I think showing the above plus whatever arguments strike you from the link I sent you should do the trick. Also, to assuage your worries about the normalizing constant, see this paper by Geyer. He demonstrates that as long as the log-likehood approaches a quadratic, then the underlying variable is asymptotically normal...no CLT or LLN needed. It's based on Le Cam's local mixed normality and direct analysis of the log-likehood function.

Related Question