$E[1/\|x\|^4]$ where $x\sim$ Gaussian($0,C\cdot $diag($1,1/2,1/3,1/4,\ldots$))

integrationnormal distributionprobability theorystatistics

  • Let $\Sigma_d$ be a diagonal matrix with diagonal $1,1/2,1/3,\ldots,1/d$
  • Let $E_d$ denote expectation w.r.t normally distributed $x$ which is centered at 0 with covariance matrix $\frac{\Sigma_d}{\operatorname{Tr}\Sigma_d}$

What is the value of the following limit?
$$\lim_{d\to \infty} E_d\left[\frac{1}{\|x\|^4}\right]$$

You can show that $\lim_{d\to\infty} E_d[\|x\|^4]=1$ by using Gaussian fourth-moment formula $E[\|x\|^4]=(\operatorname{Tr}\Sigma)^2+2\operatorname{Tr}(\Sigma^2)$

Numerical simulation suggests the limit in question might be equal to $1$, any tips?

enter image description here

notebook

Closely related issue was discussed on stats.SE and on Mathoverflow in the last month. Question is still open.

Best Answer

Let $$\frac{\Sigma_d}{\operatorname{Tr}\Sigma_d} = \mathrm{diag}(a_1, a_2, \cdots, a_d).$$

By Jensen's inequality, we have $$\mathbb{E}\left[\frac{1}{\|x\|^4}\right] \ge \frac{1}{(\mathbb{E}[\|x\|^2])^2} = 1. \tag{1}$$

Using the known identity ($q > 0$) $$\frac{1}{q^2} = \int_0^\infty t\, \mathrm{e}^{-tq} \, \mathrm{d} t,$$ we have \begin{align*} \mathbb{E}\left[\frac{1}{\|x\|^4}\right] &= \mathbb{E}\left[ \int_0^\infty t\,\mathrm{e}^{-t\|x\|^2} \,\mathrm{d} t \right]\\ &= \int_0^\infty t\, \mathbb{E}[\mathrm{e}^{-t\|x\|^2}]\, \mathrm{d} t \\ &= \int_0^\infty t\, \prod_{i=1}^d \mathbb{E}[\mathrm{e}^{-tx_i^2}]\, \mathrm{d} t\\ &= \int_0^\infty t\, \prod_{i=1}^d \frac{1}{\sqrt{1 + 2a_i t}} \,\mathrm{d} t \\ &= \int_0^\infty t\, \mathrm{e}^{-\frac12\sum_{i=1}^d \ln (1 + 2a_i t)} \,\mathrm{d} t \tag{2} \end{align*} where we use $\mathbb{E}[\mathrm{e}^{-tx_i^2}] = \int_{-\infty}^\infty \mathrm{e}^{-ty^2}\cdot \frac{1}{\sqrt{2\pi a_i}}\mathrm{e}^{-\frac{y^2}{2a_i}}\,\mathrm{d} y = \frac{1}{\sqrt{1 + 2a_i t}}$.

Fact 1: $\ln(1 + 2a_i t) \ge \frac{1}{i}\ln (1 + 2a_1 t)$ for all $i$ and all $t \ge 0$.
(Proof: Note that $a_i = a_1/i$. Let $f(t) = \ln(1 + 2a_i t) - \frac{1}{i}\ln (1 + 2a_1 t)$. We have $f'(t) = \frac{4a_1^2 t (i - 1)}{i(2a_1t + i)(2a_1 t + 1)} \ge 0$. Also, $f(0) = 0$. The desired result follows.)

Denote $H_d = \sum_{i=1}^d \frac{1}{i}$. By Fact 1, we have $$-\frac12\sum_{i=1}^d \ln (1 + 2a_i t) \le -\frac12 \ln(1 + 2a_1 t) \cdot \sum_{i=1}^d \frac{1}{i} \le - \frac12 H_d\ln(1 + 2t/H_d). \tag{3}$$

From (3), we have $$ \int_0^\infty t\, \mathrm{e}^{-\frac12\sum_{i=1}^d \ln (1 + 2a_i t)} \,\mathrm{d} t \le \int_0^\infty t\, \mathrm{e}^{- \frac12 H_d\ln(1 + 2t/H_d)} \,\mathrm{d} t. \tag{4} $$

From (1) and (2) and (4), we have $$1\le \mathbb{E}\left[\frac{1}{\|x\|^4}\right] \le \int_0^\infty t\, \mathrm{e}^{- \frac12 H_d\ln(1 + 2t/H_d)} \,\mathrm{d} t. $$

By the Dominated Convergence Theorem, we have $$\lim_{d\to \infty} \int_0^\infty t\, \mathrm{e}^{- \frac12 H_d\ln(1 + 2t/H_d)} \,\mathrm{d} t = 1.$$ (Note: Note that $u \mapsto u \ln (1 + 2t/u)$ is non-decreasing on $u > 0$. Let $f_d(t) = t\, \mathrm{e}^{- \frac12 H_d\ln(1 + 2t/H_d)}$. Then $f_2(t) \ge f_3(t) \ge \cdots $, and $\lim_{d\to \infty} f_d(t) = t\mathrm{e}^{-t}$, and $\int_0^\infty f_m(t)\,\mathrm{d} t < \infty$ for some $m$. See: Monotone Convergence theorem for decreasing sequence)

Thus, we have $$\lim_{d\to \infty} \mathbb{E}\left[\frac{1}{\|x\|^4}\right] = 1.$$

We are done.