As this is just a reference request, I hope I can get away with being brief. I am simply wondering about generalizations of the classical strong law of large numbers, which states that
$$n^{-1}\sum_{i=1}^n X_i \to EX$$
almost surely whenever $(X_n)$ is an iid sequence of integrable random variables.
What can we say about convergence of sample mean to expected value without the iid assumption? In other words, how can we weaken the assumption of iid and still get the desired convergence?
Best Answer
One can relax the independent identically distributed assumption in the following ways.
Assume that the sequence $\left(X_i\right)_{i\geqslant 0}$ is strictly stationary, that is, for all $N$, the vectors $\left(X_i\right)_{i=1}^n$ has the same distribution as $\left(X_{i+1}\right)_{i=1}^n$. This implies in particular that $X_i$ has the same distribution as $X_1$ for all $i$. Ergodic theorem tells that if $X_1$ is integrable, then $\sum_{i=1}^nX_i/n\to \mathbb E\left[X_1\mid\mathcal I\right]$ almost surely, where $\mathcal I$ is the $\sigma$-algebra of invariant sets: we represent $(X_i)_{i\geqslant 0}$ as $(f\circ T^i)_{i\geqslant 0}$ where $T$ is measure preserving and $\mathcal I=\{A\mid T^{-1}A=A\}$.
An other way to relax the i.i.d. assumption is to work with martingales. In the case of identically distributed case (which is less restrictive than strict stationarity) martingale differences, one needs the assumption of integrability of $\left\lvert X_1\right\rvert\log\left(1+\left\lvert X_1\right\rvert\right) $ (see this paper by J. Elton).
Some other useful results on pairwise dependent random variables can be found in this paper by V. Korchevsky and the references therein.