Many different characterizations of sub-Gaussianity for a centred r.v. X exist including the "Laplace formulation" where $\exists \sigma > 0$ such that
$$ \mathbb{E} \exp(tX) \le \exp(t^2 \sigma^2 / 2) $$
for all $t \in \Re$, and the Orlicz norm formulation, that $\|X\|_{\psi_2} < \infty$ in
$$ \|X\|_{\psi_2} := \inf \{t > 0 : \mathbb{E} \exp(X^2 / t^2) \le 2 \}. $$
I am interested in the relationship between $\sigma$ and $\|X\|_{\psi_2}$, particularly as applied to tail bounds. Clearly this is not a bijection, because for $Z \sim N(0, 1)$ we have $\sigma_Z = 1$ and $\|Z\|_{\psi_2} = \sqrt{8/3} \approx 1.63$ by direct calculation (as setting $\mathbb{E}\exp(Z^2/t^2) = (1-2/t^2)^{-\frac12} = 2$ gives $t^2 = 8/3$), while for $Y$ with a Rademacher distribution $\sigma_Y = 1$ but $\|Y\|_{\psi_2} = 1/\sqrt{\log 2} \approx 1.20$. Thus the Orlicz norm gives Chernoff bounds that are weaker and tighter respectively than those obtained through the Laplace formulation.
Based on the equivalences reported in Wainwright (High Dimensional Statistics, p.47 or see p.39 in the draft here: https://www.stat.berkeley.edu/~mjwain/stat210b/Chap2_TailBounds_Jan22_2015.pdf, implication I to IV) we have (since the Laplace condition implies $E \exp(s X^2 / 2\sigma^2) \le (1-s)^{-\frac12}$) the upper bound
$$ \|X\|_{\psi_2} \le \sqrt{\frac{8}{3}} \, \sigma. $$
which is tight for $Z$, i.e. a Gaussian r.v. maximizes the sub-Gaussian norm for fixed $\sigma$.
My question is if there is a reversed inequality, of the form
$$ \sigma \le C \|X\|_{\psi_2} $$
and I am interested in the tightest possible numerical value of this constant (if it exists). Any other comments on the relationship between these formulations would also be much appreciated.
I have tried manipulating the equivalences of sub-Gaussianity reported in Wainwright and Vershynin (High Dimensional Probability), but these appear to lead to upper bounds rather than the lower bounds I'm interested in.
Best Answer
Based on the course notes for "Statistics and optimization in high dimensions" by Paul Sabatier found here I was able to derive the following, which is an improvement on the result shown there. $\newcommand{\E}{\mathbb{E}}$
Let $\E X=0$ and assume there is a $\theta > 0$ such that $\E e^{X^2/\theta} \le 2$. Then \begin{align*} \E[e^{tX}] &= 1 + \E \left[\sum_{k=2}^\infty \frac{(tX)^k}{k!}\right] && \text{since } \E X = 0 \\ &\le 1 + \E \left[\frac{(tX)^2}{2} \sum_{k=2}^\infty \frac{(tX)^{k-2}}{(k-2)!}\right] && 1/k! \le \frac12 (k-2)! \text{ for } k \ge 2\\ &= 1 + \frac{t^2}{2} \E \left[X^2 e^{tX} \right] \\ &\le 1 + \frac{t^2}{2} \E \left[X^2 e^{\frac12 t^2 \theta} e^{\frac{X^2}{2\theta}}\right] && \text{Young's inequality: } tX \le \frac{\theta t^2}{2} + \frac{X^2}{2\theta}\\ &= 1 + \frac12 t^2 \theta \, e^{\frac12 t^2 \theta} \; \E\left[\frac{X^2}{\theta} e^{\frac{X^2}{2\theta}}\right] \\ &\le 1 + \frac12 t^2 \theta \, e^{\frac12 t^2 \theta} \; \E\left[e^{\frac{X^2}{\theta}}\right] && z \le e^{z/2}\\ &\le 1 + t^2 \theta \, e^{\frac12 t^2 \theta} && 1 + xe^{x/2} \le e^x \text{ for } x > 0 \tag{$\star$}\\ &\le e^{t^2 \theta}. \end{align*}
Thus if $\sigma_\star$ is the tightest possible $\sigma$ in the Laplace formulation given above, we have (for a centered $X$):
$$\boxed{\sigma_\star \le \sqrt{2} \|X\|_{\psi_2}.}$$
I here prove $(\star)$ by Taylor expanding:
\begin{align*} e^x - 1 - x e^{x/2} &= \sum_{k=0}^\infty \frac{x^k}{k!} - 1 - x \sum_{k=0}^\infty \frac{x^k}{2^k k!} \\ &= \sum_{k=1}^\infty \frac{x^k}{k!} - \sum_{k=0}^\infty \frac{x^{k+1}}{2^k k!} \\ &= \sum_{k=0}^\infty \frac{x^{k+1}}{(k+1)!} - \sum_{k=0}^\infty \frac{x^{k+1}}{2^k k!} && \text{relabel indices} \\ &= \sum_{k=0}^\infty x^{k+1} \left(\frac{1}{(k+1)!} - \frac{1}{2^k k!} \right) \end{align*} $k+1 \le 2^k \implies (k+1)! \le 2^k k! \implies \frac{1}{(k+1)!} - \frac{1}{2^k k!} \ge 0$ so if $x \ge 0$ we have the conclusion.
This bound is an improvement on any others I have been able to find in the literature, but it is still unclear whether it is the tightest possible or could be improved further.