Here is what Wikipedia says about the Strong Law of Large Numbers (SLNN):
Let $X_1,X_2,X_3,\ldots$ be a sequence of independent and identically distributed random variables with $E(X_i)=\mu$, and
$$\bar{X}_n:=\frac{X_1+\cdots+X_n}{n}.$$
The Strong Law of Large Numbers (SLNN) states that $\bar{X}_n$ converges almost surely to $\mu$, i.e.,
$$P(\lim_{n\to\infty}\bar{X}_n=\mu)=1.$$
Consider an experiment of tossing a fair coin. Let $\bar{x}_n$ be the proportion of the number of heads in the first $n$ tosses. SLNN seems to imply the following:
The probability of $\lim_{n\to\infty}\bar{x}_n=0.5$ is 1.
But how can one calculate the "probability that the numerical sequence $\{\bar{x}_n\}$ converges to 0.5?" (Forgive me if this is a dumb question; I'm new in probability.) I thought along the following line:
- Maybe the sample space $\Omega$ is the set of all sequences like THTHHT$\cdots$. But such $\Omega$ is infinite, so I don't know how to assign a probability to the subset of $\Omega$ consisting of those sequences giving $\lim_{n\to\infty}\bar{x}_n=0.5$. Does one "divide" the cardinality of such subset by the cardinality of $\Omega$ ? (I'm also not too familiar with measure theory.)
Best Answer
There is a way to interpret this probability. So think of $w \in \Omega$ as an infinite sequence of coin tosses. So each $\bar X_n(w)$ is average of nth tosses for this specific infinite sequence $w$.
Now, if for example $P(\lim_{n\to\infty}\bar X_n = 0.5)=1$, this means that there exists a set $A \subset \mathcal F$, with $P(A)=1$, where $\mathcal F$ is the sigma-algebra of the probability space. And for every $w\in A$, you have that $\lim_{n\to \infty} \bar X_n(w)= 0.5$.
In other words, for a given $\epsilon>0$, take an infinite sequence $w\in A$. Then, there exists an $N$ such that the sequence of $(\bar X_n(w))$ will fall inside the interval $(0.5 -\epsilon,0.5+ \epsilon)$.
Another way to think of $P(\lim_{n\to\infty}\bar X_n=0.5)$ is the “percentage” of sequences that converge to $0.5$. Note that if you have infinite possible sequences (paths), even if a finite number of them don’t converge to $0.5$, if we assume they are equally likely, then $P(\lim_{n\to\infty}\bar X_n=0.5)$ is still equal to $1$. Now, if one fourth of them converge to $0.5$, then $P(\lim_{n\to\infty}\bar X_n=0.5)=1/4$. And so on.