My goal is to show the following statement without using sub-exponential or sub-Gaussian norm;
Let $X$ be a zero-mean sub-Gaussian random variable with the variance proxy $\sigma^2$. Then $X^2$ is sub-exponential with parameters $(\nu, \alpha) = (16\sigma^2, 16\sigma^2)$.
For reference, the original material is on this site (complete lecture note) in lemma 1.12.
Also, $X \in SG(\sigma^2)$ with $E[X] = \mu$ means sub-Gaussian and variance proxy $\sigma^2$;
$$E[e^{t(X-\mu)}] \le e^{\frac{\sigma^2 t^2}{2}}, \forall t \in \mathbb R,$$
and $Z \in SE(\nu, \alpha)$ with $E[z] = \theta$ means sub-exponential with nonnegative parameter $(\nu, \alpha)$;
$$E[e^{t(Z-\theta)}] \le e^{\frac{\nu^2 t^2}{2}}, \forall |t| < 1/\alpha.$$
Here is the partial proof;
Let $Z = X^2 – E[X^2]$ for simplicity. Then, for $s \in \mathbb R$,
$$e^{sZ} = \sum_{k = 0}^{\infty} \frac{(sZ)^k}{k!} = 1 + \frac{sZ}{1!} + \frac{(sZ)^2}{2!} + …$$
Using this property, the proof goes as follows;
$$\begin{align}
E[e^{sZ}] &= 1 + 0 + \sum_{k = 2}^{\infty} \frac{s^kE[Z^k]}{k!} \sim (1) \\
&= 1 + \sum_{k = 2}^{\infty} \frac{s^kE[(X^2 – E[X^2])^k]}{k!} \sim (2) \\
& \le 1 + \sum_{k = 2}^{\infty} \frac{s^k2^{k−1}(E[X^{2k}]+(E[X^2])^k)}{k!} \sim (3) \\
& \le 1 + \sum_{k = 2}^{\infty} \frac{s^k4^kE[X^{2k}]}{2(k!)} \sim (4)\\
& \le 1 + \sum_{k = 2}^{\infty} \frac{s^k4^k2(2\sigma^2)^k k!}{2(k!)} \sim (5)\\
&= 1 + (8s\sigma^2)^2\sum_{k = 0}^{\infty} (8s\sigma^2)^k \sim (6) \\
&= 1 + 128s^2\sigma^4 \sim (7)\\
& \le exp(128s^2\sigma^4) \sim (8)\\
\end{align}
$$
for $ |s| \le \frac{1}{16\sigma^2} $.
Now, let me explain what I can't understand for sure.
It claims that (2) -> (3) holds due to the Jensen's inequality, but I can't obtain the inequality using Jensen.
(for reference, (4) -> (5) holds due to the property that if $X \in SG(\sigma^2)$, then $E[|X|^k] \le (2\sigma^2)^{k/2}k\Gamma(k/2)$, which is on the lemma 1.4 of the site)
Any help about this proof would be grateful. Thank you.
Best Answer
I cannot find a proof, but with the extra hypothesis $E(X^2)=\sigma^2$ everything is easy. If $2s\sigma^2<1$ we can write $$E(e^{tX^2})=E\left(\int_{-\infty}^{\infty}e^{sX-\frac{s^2}{2t}}\frac{1}{\sqrt{2\pi t}}ds\right)\leq \int_{-\infty}^{\infty}e^{\frac{s^2\sigma^2}{2}-\frac{s^2}{2t}}\frac{1}{\sqrt{2\pi t}}ds=\frac{1}{\sqrt{1-2t\sigma^2}}$$ Since we have to prove that $E(e^{tX^2})\leq e^{tE(X^2)+K\sigma^4t^2}$, denoting $u=2\sigma^2t$ and satisfing $|u|<1/8$, it is easy to do with $K=2/7$ (and not $128$). To see this we have to show that $$\frac{1}{2}\log(\frac{1}{1-u})= \frac{u}{2}+\frac{u^2}{2}\sum_{n=0}^{\infty}\frac{u^n}{n+2}\leq \frac{u}{2}+Ku^2$$ or $$\frac{1}{2}\sum_{n=0}^{\infty}\frac{u^n}{n+2}\leq K$$ Finally $$\frac{1}{2}\left|\sum_{n=0}^{\infty}\frac{u^n}{n+2}\right| \leq \frac{1}{4}\sum_{n=0}^{\infty}\frac{1}{8^n}=\frac{2}{7}.$$