Variance – Law of Large Numbers for Distributions with Infinite Variance Explained

central limit theoremlaw-of-large-numberspower lawvariance

This is a purely explorative question.

I asked a question here about a "central limit theorem" for random variables with infinite variance.

I did not expect it, but it turns out that even some random variables with infinite variance nevertheless converge in distribution to a "stable distribution" (although not the normal distribution).

This makes me wonder about something which seems even more implausible: Is there a kind of "law of large numbers" for random variables with an infinite variance?

It does not necessarily have to state that the sample mean converges in probability to the expectation, but perhaps there is something that comes close.

Best Answer

Not only the (weak and strong) laws of large numbers hold for averages of iid random variables $X_i$ with no assumption on the existence of a variance, but the weak [and only the weak] law of large numbers may even occur without the existence of an expectation of $X_i$, as detailed on the corresponding Wikipedia page.

Remember that the weak law of large numbers states that there exists $\mu$ such that$$\lim_{n\to\infty}\mathbb{P}(|\bar{X}_n-\mu|>\epsilon)=0$$for every positive constant $\epsilon>0$. And that the strong law of large numbers states [informally] that $$\mathbb{P}(\lim_{n\to\infty}\bar{X}_n=\mu)=1$$In the event the expectation $\mathbb{E}[X_i]$ exists, then $\mathbb{E}[X_i]=\mu$.

Counter-examples provided in Wikipedia are

  1. $X=\sin(Z)\exp\{Z\}/Z$ when $Z\sim\mathcal{E}xp(1)$, with $\mu=\pi/2$
  2. $X=2^Z(-1)^Z/z$ when $Z\sim\mathcal{G}(1/2)$, with $\mu=-\log(2)$
  3. $X\sim F(x)$ with $$F(x)=\mathbb{I}_{x\ge e}-\frac{e\mathbb{I}_{x\ge e}}{2x\log(x)}-\frac{e\mathbb{I}_{x\le-e}}{2x\log(-x)}+\frac{\mathbb{I}_{-e\le x\le e}}{2}$$with $\mu=0$

[in the sense that the rv's have no expectation but there exits a limit $\mu$ for the sample mean].

Related Question