Solved – Bound on moment generating function

moment-generating-functionprobabilityprobability-inequalities

This question arises from the one asked here about a bound on moment generating functions (MGFs).

Suppose $X$ is a bounded zero-mean random variable taking on values in
$[-\sigma, \sigma]$ and let $G(t) = E[e^{tX}]$ be its MGF. From a bound used in a proof of Hoeffding's Inequality, we have that
$$G(t) = E[e^{tX}] \leq e^{\sigma^2t^2/2}$$
where the right side is recognizable as the MGF
of a zero-mean normal random variable with standard deviation $\sigma$.
Now, the standard deviation of $X$ can be no larger than $\sigma$, with
the maximum value occurring when $X$ is a discrete random variable such that $P\{X = \sigma\} = P\{X = -\sigma\} = \frac{1}{2}$. So, the bound referred
to can be thought of as saying that the MGF of a zero-mean bounded random
variable $X$ is bounded above by the MGF of a zero-mean normal random
variable whose standard deviation equals the maximum possible standard
deviation that $X$ can have.

My question is: is this a well-known result of independent interest
that is used in places other than in the proof of Hoeffding's Inequality,
and if so, is it also known to extend to random variables with nonzero means?

The result that prompts this question allows asymmetric range $[a,b]$ for $X$ with $a < 0 < b$ but does
insist on $E[X] = 0$. The bound is
$$G(t) \leq e^{t^2(b-a)^2/8} = e^{t^2\sigma_{max}^2/2}$$
where $\sigma_{\max} = (b-a)/2$ is the maximum standard deviation possible
for a random variable with values restricted to $[a,b]$,
but this maximum is not attained by zero-mean random variables unless
$b = -a$.

Best Answer

I can't answer the first part of your question, but as for extending it to random variables with nonzero means...

First, note that any r.v. $Z$ with finite range $[a+\mu,b+\mu]$ and (necessarily finite) mean $\mu$ can be transformed into an r.v. $X = Z-\mu$ that is, of course, zero mean with range $[a,b]$ (thus satisfying the conditions in your problem statement). The transformed variate has m.g.f. $\phi_X(t) = \exp\{-\mu t\}\phi_Z(t)$ (by basic properties of the m.g.f.) Multiplying both sides by $\exp\{\mu t\}$ and applying the inequality gives:

$\phi_Z(t) = \exp\{\mu t\}\phi_X(t) \leq \exp\{\mu t\}\exp\{t^2\sigma^2_\text{max}/2\} = \exp\{\mu t + t^2\sigma^2_\text{max}/2\} $

Not surprisingly, the m.g.f. of a Normal random variable with the same mean and standard deviation equal to $\sigma_\text{max}$.

Related Question