In the appendix of this paper appendix, Lemma I.4 the following Gaussian concentration lemma is given.
Consider $d$– dimensional multivariate Gaussian distribution $\overline{\xi}_t^k \sim \mathcal{N} (0, H \nu_k(\delta)(\Sigma^t_k)^{-1})$. For any $\delta > 0$, with probability $1-\delta$
\begin{equation}
\|\bar{\xi}_t^k\|_{\Sigma_t^k} \leq c\sqrt{H d \nu_k(\delta)\log(d/\delta)}
\end{equation}
where c is some absolute constant.
Here $\Sigma^t_k$ is a positive definite matrix and $\nu_k$ is a function of $\delta$. In the paper they use a specific definition for $\nu_k$ but from the lemma description, it seems this lemma comes from more general statement. Could someone point me to a reference where I can find a more general statement of this? My guess there is something of the following form:
Consider random variable $X \sim \mathcal{N}(0,\alpha(\delta) \Sigma)$ where $\alpha$ is a scalar function of $\delta$ and $\Sigma \in \mathbb{R}^{d\times d}$. Then for any $\delta > 0$, with probability $1-\delta$
\begin{equation}
\|X\|_{\Sigma^{-1}} \leq c\sqrt{d \alpha(\delta)\log(d/\delta)}
\end{equation}
where c is some absolute constant.
Best Answer
If $G$ is a random vector with $d$ iid $N(0,1)$ entries then $P(|\|G\|\le E[\|G\|] + \sqrt{2\log(1/\delta)} ) \ge 1-\delta $; this follows by Gaussian concentration of $\|G\|$ which is a 1-Lipschitz of the entries of $G$, see for instance Theorems 5.5, 5.6 in Concentration Inequalities: A Nonasymptotic Theory of Independence by Boucheron Gabor Lugosi and Massart. Furthermore $E[\|G\|]\le \sqrt d$ by Jensen's inequality.
Now in your problem you may apply this result to $G=E[XX^T]^{-1/2} X$ where $E[XX^T]=\Sigma \alpha(\delta)$. It looks that the above concentration bound gives $P\Big(|\|\Sigma^{-1/2}X\|\le \sqrt{\alpha(\delta)}\big( \sqrt{d}+ \sqrt{2\log(1/\delta) }\big) \Big) \ge 1-\delta $.
Section 3.1 of the book https://www.math.uci.edu/~rvershyn/papers/HDP-book/HDP-book.html by Vershynin is also relevant.