Suppose that $X_1, X_2, \ldots, $ is a sequence of iid random variables with $\mathbb{E}(X_i) = \mu$ and $Var(X_i) = \sigma^2 < \infty$. Define $S_n = \sum_{i=1}^{n}X_i$.
Then as $n$ approaches infinity, we have that $\sqrt{n}(S_n – \mu)$ converges in distribution to a normal $N(0,\sigma^2)$ distribution:
$$
\sqrt{n}\left(\left(\frac{1}{n}\sum_{i=1}^n X_i\right) – \mu\right)\ \xrightarrow{d}\ N\left(0,\sigma^2\right).
$$
Assuming that $\sigma>0$, convergence in distribution is taken to be that the cumulative distribution functions of $\sqrt{n}(S_n – \mu)$ converge pointwise to the cdf of the $N(0, \sigma^2)$ distribution in that for every real number $z$:
$$
\lim_{n\to\infty} \Pr\left[\sqrt{n}(S_n-\mu) \le z\right] = \Phi\left(\frac{z}{\sigma}\right) ,
$$
My Question: I am having a hard time understanding what is means when they say the above convergence happens with rate $\sqrt{n}$.
The notion of rate implies that it characterizes how fast something converges, but I am failing to see in the equation above where the slowness or the "fastness" is coming in. Can someone help me see this?
Best Answer
The mean of $S_n$ is $n\mu$ and the variance of $S_n$ is $n\sigma^2$ so the standard deviation of $S_n$ is $\sqrt{n}\sigma$
This has the consequence that $\dfrac{S_n}{n}$ has mean $\mu$ and variance $\dfrac{\sigma^2}{n}$ and standard deviation $\dfrac{\sigma}{\sqrt{n}}$. I suspect that this is what led to your phrase "convergence happens with rate $\sqrt{n}$"
So defining $T_n = \dfrac{S_n-n\mu}{\sqrt{n}\sigma}$, you can find $T_n$ has mean $0$ and variance $1$ and standard deviation $1$
What the central limit theorem says is that this converges in distribution to a standard normal as $n$ increases, i.e
$$\lim_{n\to\infty} \Pr\left[T_n \le t\right] = \Phi\left(t\right)$$ or $$\lim_{n\to\infty} \Pr\left[\dfrac{S_n-n\mu}{\sqrt{n}\sigma} \le t\right] = \Phi\left(t\right)$$ or $$\lim_{n\to\infty} \Pr\left[\sqrt{n}\dfrac{\frac{S_n}{n}-\mu}{\sigma} \le t\right] = \Phi\left(t\right)$$ and letting $z=t\sigma$ $$\lim_{n\to\infty} \Pr\left[\dfrac{S_n-n\mu}{\sqrt{n}} \le z\right] = \Phi\left(\frac{z}{\sigma}\right)$$ or $$\lim_{n\to\infty} \Pr\left[\sqrt{n}\left(\frac{S_n}{n}-\mu\right)\le z\right] = \Phi\left(\frac{z}{\sigma}\right)$$
which is not quite what you have written, as it is a description of the behaviour of $\frac{S_n}{n}$