I am currently trying to learn about conditional expectation and I have the following textbook problem I try to solve.
Problem
Fred rolls a die and observes the outcome. He tells Gretel and Hansel if the number of eyes is odd or even. He also tells Gretel if the number of eyes is greater or smaller than 3.5 . He then asks Gretel and Hansel to estimate the number of eyes. For each outcome, what is Gretel's estimate? What is Hansel's estimate? What is Hansel's estimate of Gretel's estimate? What is Gretel's estimate of Hansel's estimate?
My current attempt
I will do everything quite explicit so you understand my current level within measure theory/probability theory.
We start with defining the needed terms. In our case the universe is $\Omega = \{1,2,3,4,5,6\}$ and sigma algebra $\mathcal{F}$ is the power set of $\Omega$ and we have the probability measure $P(\omega) = 1/6$ for each $\omega \in \Omega$, so we now have the measure space $(\Omega,\mathcal{F},P)$. The random variable we are observing is $X(\omega) = \omega$. Let us determine the information/sigma-algebra from Hansel and Gretel's perspective.
From my understanding no information would correspond to the sigma algebra $\sigma_0 = \{\emptyset,\Omega\}$. Hansel's sigma algebra is
\begin{align*}
\mathcal{H} = \{\emptyset,\Omega,\{1,3,5\},\{2,4,6\} \} \ .
\end{align*}
Gretel's sigma algebra is
\begin{align*}
\mathcal{G} = \{\emptyset,\Omega,\{1,3\}, \{5\}, \{1,2,3\},\{2,4,6\},\{2,4,5,6\}, \{1,3,2,4,6\}, \{2\}, \{4,6\}, \{1,3,5\}, \{1,2,3,5\}, \{1,3,4,5,6\}\} \ .
\end{align*}
We see that $\sigma_0 \subseteq \mathcal{H} \subseteq \mathcal{G} \subseteq \mathcal{F}$ so Gretel pocess more information than Hansel.
The definition I have of conditional expectation is:
Let there be given a random variable $X$ on $(\Omega, \mathcal{F}, P)$ such that $E|X|<\infty$ and a sub-sigma-algebra $\mathcal{A} \subseteq \mathcal{F}$. The conditional expectation of $X$ with respect to $\mathcal{A}$ is the almost surely unique random variable $Z = E\{X|\mathcal{A}\}$ which is measurable with respect to $\mathcal{A}$ and for which
\begin{align*}
E\{Z \cdot 1_A\} = E\{X \cdot 1_A\}
\end{align*}
for any $A \in \mathcal{A}$. Where $1_A$ is the indicator function.
In my problem we have a discrete expectation value and the expectation value of $X(\omega) = \omega$ is finite:
\begin{align*}
E|X|= \int |X| dP = \int \omega dP = \sum_{\omega \in \Omega} \omega P(\omega) \leq 6 \sum_{\omega \in \Omega} P(\omega) = 6 < \infty
\end{align*}
Where we have used $\omega \leq 6$ for all $\omega \in \Omega$.
From my understanding of the definition of conditional expectation we have to find the random variable $Z$ which fulfill
\begin{align*}
\sum_{\omega \in A} Z(\omega)P(\omega) = \sum_{\omega \in A} X(\omega) P(\omega) = \sum_{\omega \in A} \omega P(\omega)
\end{align*}
for any $A \in \mathcal{A}$. And we call this random variable $Z$ for $E\{X|\mathcal{A}\}$. So if I want to find $Z$ for Gretel we put $\mathcal{A} = \mathcal{G}$, but I am confused about how I can use this to anything? It seems like I will always conclude that $Z = \omega$?
Best Answer
I assume that you're familiar with the usual way to do answer these questions (by developing conditional distributions using Bayes' rule, as covered in a standard non-measure theoretic probability class), and want to do this systematically to develop understanding for formal definition of conditional expectation. This is definitely a useful exercise to do at least once. However, on the off chance that you are not familiar with these basic techniques, I'd strongly recommend reviewing them from an undergrad probability book first, because the following is overly cumbersome for the task at hand.
So, let's work out $Z =\mathbb{E}[X|\mathcal{G}]$ closely. Since the state space is finite, this amounts to finding the six values $Z(\omega)$ for $\omega \in [1:6].$ Now, we know that $Z$ has to be $\mathcal{G}$-measurable. So, for any value $v$, the set $\{\omega: Z(\omega) = v\}$ must lie in $\mathcal{G}$. [1]
We first use this measurability condition to determine the structure of $Z$. For the sake of contradiction, suppose that $Z(1) \neq Z(3)$. Then the set $\{\omega: Z(\omega) = Z(1)\}$ does not contain $3$, but contains $1$. But any set in $\mathcal{G}$ that has a $1$ also has a $3$, giving a contradiction. This means that $Z(1) = Z(3)$. Similarly, we have $Z(4) = Z(6)$.
(More generally - the point is that $\mathcal{G}$ is generated by the four sets $G_1 = \{1,3\}, G_2 = \{2\}, G_3 = \{4,6\}, G_4 = \{5\}$. So, any random variable measurable with respect to it must be a simple function of the form $\sum a_i \mathbf{1}_{G_i}.$ You should try to show this.)
Now, taking various choices of $G \in \mathcal{G},$ we can further resolve the values of $Z$ using the expectation properties. For instance, note that for $G = \{2\},$ $$ \mathbb{E}[Z\mathbf{1}_{\{2\}}] = \mathbb{E}[X\mathbf{1}_{\{2\}}] \iff Z(2) P(2) = X(2) P(2) \iff Z(2) = X(2).$$
Similarly, taking $G = \{1,3\},$ we find that $$ Z(1)P(1) + Z(3)P(3) = Z(1)(P(1) + P(3)) = X(1)P(1) + X(3) P(3) \\ \iff Z(1) = \frac{X(1) P(1) + X(3) P(3)}{P(1) + P(3)}, $$ where we have used our previous observation that $Z(1) = Z(3)$.
As an exercise, set up similar calculations to determine $Z(4), Z(5), Z(6)$.
Finally, if we evaluate the numbers by setting $X(\omega) = \omega, P(\omega) = 1/6,$ we'll find that $$ Z(1) = Z(2) = Z(3) = 2,\\ Z(4) = Z(5) = Z(6) = 5.$$
[1] More generally, we would work with Borel sets and consider $Z^{-1}(B) = \{\omega: Z(\omega) \in B\}$, but in this case the discreteness makes things easier.