Examples of the events for which we cannot assign “meaningful” probabilities

probability

Quote from the book I'm reading:

Any collection of possible outcomes, including the sample space $\Omega$ and its complement, the empty set $\emptyset$, may qualify as an event. Strictly speaking, however, some sets have to be excluded. In particular, when dealing with probabilistic models involving an uncountably infinite sample space; there are certain unusual subsets for which one cannot associate meaningful probabilities.

Question 1

  • What is meant by "meaningful" probabilities?

Question 2

  • Can you provide an example in which we cannot assign meaningful probabilities to the events of the sample space?

Best Answer

Say your sample space is $\Omega=[0,1]$, and you want want to define probabilities on the possible events $X$ according to a distribution function $F(x)$. You are thus looking for a function $\mathbb{P}$ that assigns a probability to every subset $X \subset \Omega$ such that

  1. $\mathbb{P}(X) = F(b) - F(a)$ if $X=[a,b]$

and such that the probabilities assigned by $\mathbb{P}$ are meaningful in the sense that they obey the usual laws for probabilities,

  1. $\mathbb{P}(\emptyset)=0$, $\mathbb{P}(\Omega) = 1$, $0 \leq \mathbb{P}(X) \leq 1$ for all $X \subset \Omega$,

  2. $\mathbb{P}(X_1 \cup X_2 \ldots) = \mathbb{P}(X_1) + \mathbb{P}(X_1) + \ldots$ for all disjoint sequences $X_1,X_2,\ldots \subset \Omega$ (disjoint means $X_i \cap X_j = \emptyset$ if $i \neq j$, and note that countably infinite sequences are allowed!).

Even for some very well-behaved $F$ (e.g. for $F(x)=x$, it's hard to imagine a more well-behaved function than this), this turns out to be not possible. There simply isn't a function $\mathbb{P}$ that assign each subset of $[0,1]$ a probability (i.e. a real number between 0 and 1) such that the requirements above are fulfilled.

But it is possible if we exclude certain very weird and hard to imagine sets. All of these sets require the axiom of choice to even construct them, so you may imagine them to be artifacts of mathematical set theory, and not sets that you ever want to actually compute a probability for (unless you're a set theorist, maybe). Such sets are called non measurable.

Vitali sets are examples of non measurable sets, but unfortunately their construction requires a bit of number theory.