As I learned it, a Bernoulli trial has exactly two possible outcomes (sometimes symbolized by $1$ or $0$), with each trial being probabilistically independent and each trial (or "experiment") having the same probability; and a sample space is the set of all possible outcomes; e.g. flipping a fair coin twice would yield a sample space of $\{HH, HT, TH, TT\}$.
The question: how to mathematically prove that an infinite sequence of Bernoulli trials yields a sample space of the set of all infinite binary sequences of outcomes?
It seems obvious to me that an infinite sequence of Bernoulli trials yields a sample space that is the set of all infinite binary sequences of outcomes (or "Cantor space" as I was taught), but I've interacted with some who doubt it. For sake of concreteness, I'll use the classic example of flipping a fair coin for the Bernoulli trial, such that infinitely many of these Bernoulli trials involves flipping a fair coin infinitely many times, and the apparent sample space would be every possible infinite sequence of heads/tails.
I thought of using mathematical induction to show this, but the response I got from using this was that it applies only to any finite number but not the whole set. For example, if I wanted to show that a sequence of all heads was part of the sample space via mathematical induction, at most I could prove this (where $H(x)$ means every flip less than or equal to the $x$th flip came up heads):
$ \forall x \lozenge H(x)$
And not this:
$\lozenge \forall x H(x)$
As such, I don't know to mathematically prove that an infinite sequence of Bernoulli trials yields a sample space of the set of all infinite binary sequences of outcomes. How does one do it?
To quote from this MIT open course link:
A Bernoulli process may be limited to a particular number of trials (e.g. 7 games, or 10 coin tosses), or it may go on indefinitely, in which case we may regard it as an infinite process. A finite Bernoulli sample space
consists of all binary sequences of some particular length $n$ (1 denotes
success, 0 failure). In the infinite case, the sample space consists of all infinite binary sequences.
The notion that, "In the infinite case, the sample space consists of all infinite binary sequences" is what I'm looking to prove.
Best Answer
Let $(S, \mathcal{F}, P)$ be a probability triplet: $S$ is the sample space, $\mathcal{F}$ is a sigma algebra on $S$ (containing all the events), and $P:\mathcal{F}\rightarrow \mathbb{R}$ is a probability measure.
Suppose $\{X_i\}_{i=1}^{\infty}$ is a sequence of mutually independent and identically distributed (i.i.d.) random elements. In particular $$X_i:S\rightarrow \{H, T\}$$ is a measurable map from the sample space to the set $\{H,T\}$, so for each $i \in \{1, 2, 3,...\}$ we have
$$\{\omega \in S: X_i(\omega) = H\} \in \mathcal{F}$$ Assume $P[X_i=H]=P[X_i=T]=1/2$.
Claim 1: For each sequence $\{h_i\}_{i=1}^{\infty}$ with $h_i \in \{H,T\}$, we have $$\cap_{i=1}^{\infty} \{X_i=h_i\}= \cap_{i=1}^{\infty} \{\omega \in S: X_i(\omega)=h_i\} \in \mathcal{F}$$
Proof: Since $\mathcal{F}$ is a sigma algebra, the countable intersection of events in $\mathcal{F}$ is in $\mathcal{F}$. $\Box$
Claim 2: It is possible to construct $(S, \mathcal{F}, P)$, for which such i.i.d. random elements $\{X_i\}$ exist, in these example cases:
a) $S = \{red, blue\} \cup [0,1)$
b) $S = A$, where $A=\{(h_1, h_2, h_3, ...) : h_i \in \{H,T\}\quad \forall i \in \{1, 2, 3,...\}\}$.
c) $S = A \setminus \{(T, T, T, T, ...)\}$.
In particular, examples (a) and (c) can be viewed as "counter-examples" to your claim that the sample space must contain all binary sequences of H/T. The example (c) contains all binary sequences of H/T except for the all-Tails sequence $(T, T, T, T,...)$ (just start with the probability triplet in part (b) but throw away the probability-0 outcome of all tails).
Quick justifications for (a)-(c):
a) Use $P[\{red\}]=P[\{blue\}]=0$ and choose $\omega \in [0,1)$ according to the Borel sigma algebra and Borel measure. Write $\omega\in [0,1)$ as $$ \omega = \sum_{i=1}^{\infty} \omega_i 2^{-i}$$ where $\{\omega_i\}$ is the unique binary expansion that does not contain an infinite tail of 1s. Define for each $i \in \{1,2,3,...\}$ $$X_i(red)=X_i(blue)=H$$ For $\omega \in [0,1)$ define $$X_i(\omega) = \left\{\begin{array}{cc} H & \mbox{if $\omega_i=1$}\\ T & \mbox{else} \end{array}\right.$$
b) This is the standard one.
c) Just start with (b) and throw away an outcome of probability 0.