It is given in definition, that $\sigma$ algebra is a type of algebra of sets. So if algebra of sets is defined as being closed under intersections with finitely many subsets, does that mean that being closed under countable intersections implies being closed under intersections with finitely many subsets? Countable set is a set which is bijective to a subset of natural numbers? If so, does finitely many mean that you can also have a bijective function with natural number set? And then, why is that important in probability?
The difference between closed under intersections of finitely many subsets and closed under countable intersections
probabilityprobability theory
Related Solutions
Both results are actually equivalent. You can prove one from the other.
Regarding the first result:
Let $\mathcal{C}$ be a class of subsets of $\Omega$ closed under finite intersections and containing $\Omega$. Let $\mathcal{B}$ be the smallest class containing $\mathcal{C}$ which is closed under increasing limits and by difference. Then $\mathcal{B} = \sigma ( \mathcal{C})$.
Some books call it "Monotone Class Theorem", although this is not the most usual naming.
A class having $\Omega$, closed under increasing limits and by difference is called a "Dynkin $\lambda$ system". A non-empty class closed under finite intersections is called a "Dynkin $\pi$ system".
The result above can be divided in two results
1.a. A $\lambda$ system which is also a $\pi$ system is a $\sigma$-algebra. 1.b. Given a $\pi$ system, the smallest $\lambda$ system containing it is also a $\pi$ system.
Some books call result 1.a (or result 1.b) "Dynkin $\pi$-$\lambda$ Theorem.
Some quick references is https://en.wikipedia.org/wiki/Dynkin_system
The second result
Let $G$ be an algebra of sets and define $M(G)$ to be the smallest monotone class containing $G$. Then $M(G)$ is precisely the $\sigma$-algebra generated by $G$, i.e. $\sigma(G) = M(G)$.
Where a monotone class in a set $R$ is a collection $M$ of subsets of $R$ which contains $R$ and is closed under countable monotone unions and intersections.
is usually called "Monotone Class Lemma" (or theorem) you can find it in books like Folland's Real Analysis or Halmos' Measure Theory. In fact, Halmos presents a version of this result for $\sigma$-rings.
Let $G$ be ring of sets and define $M(G)$ to be the smallest monotone class containing $G$. Then $M(G)$ is precisely the $\sigma$-ring generated by $G$.
Let us prove that the results are equivalent
Result 1: Let $\mathcal{C}$ be a class of subsets of $\Omega$ closed under finite intersections and containing $\Omega$. Let $L(\mathcal{C})$ be the smallest class containing $\mathcal{C}$ which is closed under increasing limits and by difference. Then $L(\mathcal{C}) = \sigma ( \mathcal{C})$.
Result 2: Let $G$ be an algebra of sets and define $M(G)$ to be the smallest monotone class containing $G$. Then $M(G)$ is precisely the $\sigma$-algebra generated by $G$, i.e. $\sigma(G) = M(G)$.
Where a monotone class in a set $R$ is a collection $M$ of subsets of $R$ which contains $R$ and is closed under countable monotone unions and intersections.
Proof:
(2 $\Rightarrow$ 1). Note that any class containing $\mathcal{C}$ which is closed under increasing limits and by difference is close by complement because $\Omega \in \mathcal{C}$, and so it is also closed by decreasing limits. So it is closed under countable monotone unions and intersections. It means: any class containing $\mathcal{C}$ which is closed under increasing limits and by difference is a monotone class.
Note also that any class containing $\mathcal{C}$ which is closed under increasing limits and by difference contains $A(\mathcal{C})$ the algebra generated by $\mathcal{C}$.
Then using Result 2 we have $$ \sigma(\mathcal{C}) = \sigma(A(\mathcal{C})) = M(A(\mathcal{C})) \subseteq L(A(\mathcal{C}))=L(\mathcal{C}) $$ Since $\sigma(\mathcal{C})$ is a class containing $\mathcal{C}$ which is closed under increasing limits and by difference, we have $L(\mathcal{C}) \subseteq \sigma(\mathcal{C})$, so $L(\mathcal{C}) = \sigma(\mathcal{C})$.
(1 $\Rightarrow$ 2). First let us prove that $M(G)$ is a class containing $G$ which is closed under increasing limits and by difference. Since $M(G)$ is monotone, we have that $M(G)$ is closed under increasing limits.
Now, for each $E\in M(G)$, define
$$M_E=\{ F \in M(G) : E\setminus F , F \setminus E \in M(G) \}$$
Since $M(G)$ is a monotone class, $M_E$ is a monotone class. Moreover, if $E\in G$ then for all $F \in G$, $F\in M_E$, because $G$ is an algebra. So, if $E\in G$, $G \subset M_E$. So, if $E\in G$, $M(G) \subset M_E$. It means that for all $E\in G$, and all $F \in M(G)$, $F \in M_E$. So, for all $E\in G$, and all $F \in M(G)$, $E \in M_F$. So, for all $F \in M(G)$, $G \subset M_F$, but since $M_F$ is a monotone class, we have, for all $F \in M(G)$, $M(G)\subset M_F$. But that means that $M(G)$ is closed by differences.
So we proved that $M(G)$ is a class containing $G$ which is closed under increasing limits and by difference.
So by Result 1, $$\sigma(G)=L(G) \subseteq M(G)$$ Since $\sigma(G)$ is a monotone class, we have $$ M(G) \subseteq \sigma(G)$$ So we have $$\sigma(G)= M(G)$$
Why do the axioms need countable additivity?
Let $S$ be a (nonempty) sample space and $\mathcal{F}$ a sigma-algebra of events. A function $P:\mathcal{F}\rightarrow \mathbb{R}$ is a probability measure if it satisfies these standard axioms:
$P[A]\geq 0$ for all $A \in \mathcal{F}$.
$P[S]=1$.
(finite additivity) $P[A\cup B] = P[A]+P[B]$ whenever $A, B$ are disjoint sets in $\mathcal{F}$.
(countable additivity) If $\{A_i\}_{i=1}^{\infty}$ is a sequence of disjoint sets in $\mathcal{F}$ then $$ P[\cup_{i=1}^{\infty}A_i] = \sum_{i=1}^{\infty}P[A_i]$$
Why do we require the last bullet (countable additivity)? Real analysis gives us useful definitions for the sum of a finite or countably infinite number of nonnegative terms. Thus, we want our probability theory to allow both finite and countably infinite sums. This is why countable additivity is needed. With countable additivity we can say $$ \mbox{$\{A_i\}_{i=1}^{\infty}$ disjoint events} \implies P[\cup_{i=1}^{\infty} A_i] = \sum_{i=1}^{\infty} P[A_i]$$ In particular, if we partition the sample space $S$ into a countably infinite number of disjoint pieces, we want the sum of the probabilities of each piece to be 1. It is impossible to prove this from the first three bullets: The axioms must include the countable additivity axiom.
Indeed the following example is possible: Let the sample space be the natural numbers: $$S = \mathbb{N} = \{1, 2, 3, ...\}= \cup_{i=1}^{\infty} \{i\}$$ Let the sigma-algebra be the set of all subsets of natural numbers: $$\mathcal{F}=2^{\mathbb{N}}$$ We can build a function $P:2^{\mathbb{N}}\rightarrow\mathbb{R}$ that satisfies the first 3 bullets of the above axioms (but not the countable additivity bullet) such that $$ P[\mathbb{N}] =1 \neq 0 =\sum_{i=1}^{\infty} P[\{i\}] \quad (Eq. 1)$$ This is just weird. So we need the countable additivity axiom. How to build an example that satisfies (Eq. 1)? This is very difficult and uses a theory of Banach limits, the main idea is to define $P[A]$ for each $A \subseteq \mathbb{N}$ by: $$ P[A] = \lim_{n\rightarrow\infty} \frac{|A\cap \{1, 2, ...,n\}|}{n}$$ whenever the limit exists, and to define $P[A]$ using a Banach limit when the regular limit does not exist. It is easy to see that $P[A]=0$ for every finite set $A\subseteq \mathbb{N}$ and so the weird (Eq. 1) holds. It can also be shown that the first three axiom bullets hold for this example.
What if we change sigma-algebras to only use disjoint unions?
The probability axioms only require countable disjoint unions. Why not change the sigma-algebra definition to only require that the countable disjoint union of sets in the sigma-algebra are also in the sigma-algebra?
Let $S$ be a nonempty sample space. The standard definition of a sigma-algebra on $S$ is a collection $\mathcal{F}$ of subsets of $S$ such that (i) $S$ is in $\mathcal{F}$; (ii) if $A$ is in $\mathcal{F}$ then $A^c$ is in $\mathcal{F}$; (iii) If $A_1, A_2, ...$ is a countably infinite sequence of sets in $\mathcal{F}$ then $\cup_{i=1}^{\infty}A_i$ is in $\mathcal{F}$.
Wish-list for Alternative proposed definition: Let's say a "modified" sigma-algebra on $S$ is a collection $\mathcal{F}$ of subsets of $S$ such that:
$S$ is in $\mathcal{F}$.
If $A$ is in $\mathcal{F}$ then $A^c$ is in $\mathcal{F}$.
If $A$ and $B$ are in $\mathcal{F}$ then $A \cup B$ is in $\mathcal{F}$.
If $A_1, A_2, ...$ is a countably infinite sequence of disjoint subsets of $\mathcal{F}$ then $\cup_{i=1}^{\infty} A_i$ is in $\mathcal{F}$.
Well, how does this modified wish-list change the sigma algebra? Not at all. This set of 4 points on the wish-list is equivalent to the standard definition of sigma-algebra. This is because:
Induction can prove that, for every positive integer $n$, if $A_1, ..., A_n$ are in $\mathcal{F}$ then $\cup_{i=1}^n A_i$ is in $\mathcal{F}$.
The Nap D comment shows that if $A, B$ are in $\mathcal{F}$ then $A\cap B$ is in $\mathcal{F}$.
Any countably infinite union of sets in $\mathcal{F}$ can be written as a countably infinite disjoint union via $$ \cup_{i=1}^{\infty} A_i = \cup_{i=1}^{\infty} (A_i \cap (A_1\cup...\cup A_{i-1})^c)$$
Caveat: The sigma-algebra might change if we remove the part of our wish-list that says $A\cup B$ must be in $\mathcal{F}$. A counter-example and a discussion of Dynkin systems is here: Sigma-algebra requirement 3, closed under countable unions.
Note that $A \cup B$ is on our wish-list because with it we can prove $$ \boxed{P[A\cup B] = P[A] + P[B] - P[A\cap B]}$$ which is arguably one of the most intuitive (and important) results of probability theory. Probability theory would be overly restrictive if we did not have the above boxed equation.
Best Answer
$$ \bigcap_{n=1}^\infty \left( \frac{-1} n , \frac 1 n \right) = \{0\}. $$ This is an intersection of countably many open sets, but it is not an intersection of finitely many open sets. An intersection of finitely many open sets would be open, but this set is not open. “Open” in this context would mean containing an open interval about each of its points.