[Math] The law of large numbers: How does the convergence take place? How does the “remainder” look like

convergence-divergenceprobabilityresidue-calculus

I have the independent and identically distributed random variables $X_1,X_2,\ldots$ with a finite expectation $\mu$. I also have defined $S_n = X_1 + \cdots + X_n$.

According to the law of large numbers, I already know that

$S_n/n \to \mu$ almost surely as $n\to\infty$.

However, my question is: How does the convergence take place? What is the “shape” of this convergence? For example, for a given $N>0$, how close is $S_N/N$ to $\mu$? How does the difference, or remainder/residual, look like?

Are there any results, or theorems, available that tell me these kinds of things?

Best Answer

Sometimes a weak law of large numbers gives you better quantitative information.

As a very simple example, suppose that the $X_i$ have finite variance $\sigma^2$. Then it is easy to see that $\operatorname{Var}(\frac{S_n}{n}) = \frac{\sigma^2}{n}$, and Chebyshev's inequality gives that for any $\epsilon > 0$, we have $$P\left(\left|\frac{S_n}{n}-\mu\right| > \epsilon\right) \le \frac{\sigma^2}{n \epsilon^2}.$$

Related Question