Question:
Let $X_1, X_2, \ldots, X_n$ be independent and identically distributed random variables such that: $$P(X_i = 1) = P(X_i = -1) = \frac{1}{2}$$
Derive the moment generating function of $Y_n := \sum^n_{i=1} a_iX_i$ where $a_1, \ldots, a_n$ are constants.
Then, let $a_i = \frac{1}{2^i}$ and show that $Y_n$ converges in distribution to the uniform distribution on $(-1, 1)$.
Attempt:
For the moment generating function:
$$M_{Y_n}(t) = \mathbb{E}(e^{t(a_1x_1 + … +a_nx_n)}) = \prod_{i=1}^n \mathbb{E}(e^{ta_iX_i}) = \frac{1}{2^n} \prod_{i=1}^n (e^{ta_i} + e^{-ta_i})$$
(Using $\mathbb{E}(e^{ta_iX_i}) = \frac{1}{2} (e^{ta_i} + e^{-ta_i})$.)
However, I'm not sure about the distribution function. So far, I can show that:
$Y_n := \sum^n_{i=1} \frac{1}{2^i} X_i$, and since $-1 \leq X_i \leq 1$,
$-\sum^n_{i=1} \frac{1}{2^i} \leq Y_n \leq \sum^n_{i=1} \frac{1}{2^i} \quad \Rightarrow \quad -1 \leq \lim Y_n \leq 1$.
I'm not sure how to proceed from here, because I feel like my next few lines are lacking rigor. I wrote:
Hence, $\forall x < -1, \quad \lim P(Y_n \leq x) = 0$, and
$\forall x > 1, \quad \lim P(Y_n \leq x) = 1$
Which is so far consistent with the distribution function of the uniform distribution. However, I am unsure if this is correct, and of how to tackle the case for $-1 \leq x \leq 1$.
Any suggestions and corrections would be appreciated, thank you!
Best Answer
Note : The $X_i$ defined by you, taking values $\pm 1$ with probability half is called a $\color{#e24f}{Rademacher}$ random variable.
One of the important theorems of probability theory, that allows you to show that a certain sequence converges is distribution, is the uniqueness theorem for moments. If you can show two things for a sequence $X_n$ :
The moments of $X_n$ converge to the moments of some random variable $X$.
Any random variable with those moments is uniquely determined.
Then you can conclude that $X_n \to X$ in distribution. In other words, if the MGFs of $X_n$ converge pointwise and the limit is the MGF of a unique random variable, then the random variables themselves converge in distribution.
A very simple theorem, which you can try to prove, is that any bounded distribution is determined by its moments. Therefore, the uniform random variable is determined by its moments, and therefore its MGF.
We look at $M_{Y_n}(t)= \frac 1{2^n} \prod_{i=1}^n (e^{ta_i} + e^{-ta_i})$, and where this is going to converge. Since $a_i = 2^{-i}$ we substitute to get $\frac 1{2^n}\prod_{i=1}^n (e^{t2^{-i}} +e^{-t2^{-i}})$.
At this point, we make an observation super-crucial to Rademacher random variables : $\frac{e^x + e^{-x}}{2} = \cosh(x)$, the hyperbolic cosine function. The reason why $a_i = \frac 1{2^i}$ works so well is that there's a nice identity called Euler's identity, which states that $$ \frac{\sinh(t)}{t} = \prod_{n=1}^{\infty} \cosh\left(\frac{t}{2^n}\right) $$ and one that you should remember in the context of MGF convergence.
Note that $M_{Y_n}(t) = \prod_{i=1}^n \cosh\left( \frac t{2^i}\right)$. Therefore, by Euler's identity, $M_{Y_n}(t) \to \frac{\sinh(t)}{t}$ for all $t \in \mathbb R$!
Now, what random variable has $\frac{\sinh(t)}{t}$ as its MGF? You can check that $E[e^{tY}] = \frac{\sinh(t)}{t}$ where $Y \sim U(-1,1)$. Thus we complete the problem.
An alternate way of deriving intuition (not a rigorous proof) is to let $Z_i = \frac{X_i + 1}{2}$. Then $Z_i$ are iid Bernoulli$(\frac 12)$ random variables taking values in $\{0,1\}$. In particular, if $Y$ is a random variable that is to be a limit of $Y_n$, then $\frac{Y+1}{2} = \sum_{i=1}^\infty Z_i 2^{-i}$ is precisely the binary expansion of a real number in $[0,1]$ chosen at random i.e. $\frac{Y+1}{2} \sim U(0,1)$. It follows that $Y \sim U(-1,1)$.
You can find far more details on Rademacher variables here.
I provide a standalone proof of the Euler identity I used above.
Note that $\frac{e^{-x} - e^{x}}2 = \frac{(e^{x/2} + e^{-x/2})(e^{x/2} - e^{-x/2})}{2}$ for all $x$, by taking a difference of squares. Therefore, we have : $$ \frac{\sinh(t)}{t} = \frac{e^{t}-e^{-t}}{2t} = \frac{(e^{t/2}-e^{-t/2})(e^{t/2} + e^{-t/2})}{2t} = \frac{e^{t/2} + e^{-t/2}}{2}\frac{e^{t/2} - e^{-t/2}}{t} \\ = \frac{e^{t/2} + e^{-t/2}}{2} \frac{e^{t/4} + e^{-t/4}}{2} \frac{e^{t/4} - e^{-t/4}}{t} = \cosh\left(\frac t2\right) \cosh\left(\frac t4\right) \frac{e^{t/4} - e^{-t/4}}{2^{-1}t}\\ = ... = \frac{e^{t2^{-n}} - e^{t2^{-n}}}{2^{-n+1}t} \prod_{i=1}^n \cosh\left (\frac t{2^i}\right) $$
for all $n \in \mathbb N$. So of course we may take $n \to \infty$ on the right hand side, and note that $\lim_{n \to \infty} \frac{e^{t2^{-n}} - e^{t2^{-n}}}{2^{-n+1}t} = 1$ since we may take $u = t2^n$ in that expression and get $\lim_{u \to 0}\frac{e^{u} - e^{-u}}{2u} = 1$, since that's the limit $\lim_{h \to 0} \frac{f(x+h) - f(x-h)}{2h} = f'(x)$ for $f(s) = e^s$ and $x=0$. The identity follows.