(Sub-)Gaussian Norm Concentration Inequality

concentration-of-measureprobability theorystatistics

I am looking for a proof of the following: Let $X_1 \dots, X_n $ be i.i.d. $N(\mu, \Sigma)$, then
$$
\mathbb{P} \left ( \|\bar{X}_n-\mu \|_{2} > \sqrt{\frac{\text{Tr}\Sigma}{n}}+\sqrt{\frac{2 \| \Sigma \| \log(1/\delta)}{n}} \right ) \le \delta
$$

where $\|\Sigma \|$ is the largest eigenvalue of $\Sigma$. This holds generally for any sub-Gaussian random variables. I am not sure how to get the $\| \Sigma\|$ term – I'm just looking for a hint or a good reference here.

Best Answer

It suffices to show $$P(\|X_1 - \mu\|_2 > \sqrt{\text{Tr} \Sigma} + \sqrt{2 \|\Sigma\| \log(1/\delta)}) \le \delta.$$ (Since $\bar{X}_n$ is also [sub-]Gaussian with mean $\mu$ and covariance $\Sigma/n$, replacing $X_1$ with $\bar{X}_n$ and $\Sigma$ with $\Sigma/\sqrt{n}$ gives the stated result.)

Without loss of generality, we can also assume $\mu=0$.

Theorem 6.3.2 and Exercise 6.3.5 in Vershynin's book seem relevant. Writing $X_1 = \Sigma^{1/2} Z$ where $Z \sim N(0, I)$, the statement in the exercise produces $$P(\|\Sigma^{1/2} Z\|_2 \ge C \|\Sigma^{1/2}\|_F + t) \le \exp\left(-\frac{ct^2}{\|\Sigma^{1/2}\|^2}\right)$$ where $\|\Sigma^{1/2}\|_F = \sqrt{\text{Tr}(\Sigma)}$ and $\|\Sigma^{1/2}\|^2 = \|\Sigma\|$. I think substituting $\delta = \exp\left(-\frac{ct^2}{\|\Sigma^{1/2}\|^2}\right)$ and rewriting will produce something like the first line of my post.

(Apologies for the hand-waving; just giving you a sketch and pointer since you are just asking for a hint/reference.)