I've been studying some statistics by the ways of measure theory, and came up with a problem in understanding conditional probability. The book gives the following definitions:
Let $(\Omega,\mathcal{F},\mathbb{P})$ be a probability space, $\mathcal{G} \subset \mathcal{F}$ be a $\sigma$-algebra and $A \in \mathcal{F}$. Then, we define a probability measure on $\mathcal{G}$ by the law $$\nu(G) = \mathbb{P}(A \cap G).$$ If $\mathbb{P}(G) = 0$, then $$\nu(G) = \mathbb{P}(G \cap A) \leq \mathbb{P}(G) = 0,$$ hence $\nu \ll \mathbb{P}$. We define $\mathbb{P}(A \mid \mathcal{G})$ as $d\nu/d\mathbb{P}$. This way, the conditional probability is kind of well-defined, since Radon-Nikodym derivatives are equal almost everywhere.
My problem here is: I don't understand what is the intuition behind this definition. For some help in understanding my problem, there is an important result
The function $\mathbb{P}(A \mid \mathcal{G})$ has the following properties almost everywhere:
- $0 \leq \mathbb{P}(A \mid \mathcal{G}) \leq 1$;
- $\mathbb{P}(\emptyset \mid \mathcal{G}) = 0$;
- If $\{A_n\}_{n = 1}^\infty$ is a sequence of two-by-two disjoint sets in $\mathcal{F}$, then $$\mathbb{P}\left(\bigcup_{n=1}^\infty A_n \mid \mathcal{G}\right) = \sum_{n = 1}^\infty \mathbb{P}(A_n \mid \mathcal{G}).$$
This lemma says that, for almost every point, $\mathbb{P}(A \mid \mathcal{G})$ is a probability measure in $\mathcal{F}$ (as a function of $A$). Now, I don't have a single clue about what a "field" of probability measures mean and how it is related in any way with concepts of elementary probability. Even more, is there any "real-world" applications of this concept? With real-world applications I mean like some kind of elementary statistics problem that can be solved using this, some applicable situation that might appear in a non-math context.
Best Answer
A key scenario is when we have a random variable $X:\Omega\rightarrow\mathbb{R}$ and we have $\mathcal{G}=\sigma(X)$, the sigma algebra generated by the random variable $X$, so that: $$P[A|\mathcal{G}]=P[A|X] \quad \forall A \in \mathcal{F}$$
Example 1: Suppose we play one of two teams equally likely. Let $X \in \{1,2\}$ be the random variable specifying which team we play. Suppose:
In this example we have $\sigma(X) = \{\phi, \Omega, \{X=1\}, \{X=2\}\}$. Let $Win$ be the event that we win. Then $$ P[Win|X] = \left\{\begin{array}{cc} 1/3 & \mbox{ if $X=1$} \\ 1/4 & \mbox{ if $X=2$} \end{array}\right.$$ Also, for all $A \in \mathcal{F}$ we have $$ P[A|X] = \left\{\begin{array}{cc} P[A|X=1] & \mbox{ if $X=1$} \\ P[A|X=2] & \mbox{ if $X=2$} \end{array}\right. $$ So you can see we select one of two probability measures depending on the value of $X$. Since $X$ is really $X(\omega)$ (a function of $\omega \in \Omega$) you can view $P[A|X]$ as a function of $\omega \in \Omega$. It is a $\sigma(X)$-measurable function from $\Omega$ to $\mathbb{R}$.
Example 2: Suppose $X:\Omega\rightarrow\mathbb{R}$ takes values in a finite or countably infinite set $S_X$ and $P[X=x]>0$ for all $x \in S_X$. Then $$\sigma(X) = \sigma(\{\omega\in \Omega : X(\omega)=x\}_{x\in S_X})$$ and $$ \boxed{P[A|X] = \sum_{x\in S_X}P[A|X=x]1_{\{X=x\}} \quad \forall A \in \mathcal{F}}$$ where $1_{\{X=x\}}$ is an indicator function that is 1 if $X=x$ and 0 else. So which probability measure we use depends on the value of $X(\omega)$. If $\omega_1 \in \Omega$ satisfies $X(\omega_1)=x_1$ then $$P[A|X](\omega_1)=P[A|X=x_1] \quad \forall A \in \mathcal{F}$$ You can show that this definition of $P[A|X]$ satisfies the requirements of a conditional expectation of $1_A$ given $\sigma(X)$: It is a $\sigma(X)$-measurable function from $\Omega\rightarrow\mathbb{R}$ (meaning it is a pure function of $X(\omega)$); $E[P[A|X]1_B]=E[1_{A\cap B}]$ for all $B \in \sigma(X)$.