I have a question on how to interpret conditional expectation its properties geometrically. There are two properties of conditional expectation in particular that I’m trying to interpret:
Given a probability space $(\Omega, \mathcal{G}, \mathbb{P})$
-
If $\mathcal{H} \subset \mathcal{G}$ then $\mathbb{E}[\mathbb{E}[X|\mathcal{H}] | \mathcal{G}] = \mathbb{E}[ X|\mathcal{H}]$. (tower rule)
-
If $ X \mathrel{\unicode{x2AEB}} G$ then $\mathbb{E}[X|\mathcal{G}] = \mathbb{E}[ X]$.
I interpret the tower rule geometrically using linear algebra as follows:
-
$H \subset G$ is interpreted as $\mathcal{H}$ is a subspace contained in the subspace $\mathcal{G}$.
-
$\mathbb{E}[X|\mathcal{H}]$ is the projection of $X$ onto $\mathcal{H}$.
-
So, $\mathbb{E}[\mathbb{E}[X|\mathcal{H}] | \mathcal{G}] = \mathbb{E}[ X|\mathcal{H}]$ can be interpreted as the projection of X onto $\mathcal{H}$, projected onto $\mathcal{G}$, is the same as the projection of X onto $\mathcal{H}$.
This makes sense. However, the reason I don't like this interpretation is that it is also true if I swap $\mathcal{H}$ and $\mathcal{G}$. So, this doesn't use the property that $\mathcal{H} \subset \mathcal{G}$. Can someone help me come up with a better interpretation of this?
related: Question about conditional expectation as projection
Best Answer
Let's start with a little Linear Algebra warm-up. Let $V$ be a vector space (which might be infinite dimensional, in that case we will require it to be a Hilbert space), and let $W \subset U$ be two (closed) sub-spaces of $V$. Denote by $P_W$ and $P_U$ the orthogonal projections on $W$ and $U$, respectively. It is straightforward to check $$P_W\cdot P_U = P_U\cdot P_W = P_W.$$ That is, the operators commute and their product is precisely $P_W$.
Back to the probabilistic setting. We consider the Hilbert space $L^2(\mathcal{G})$ of square integrable random variables measureable with respect to $\mathcal{G}$. Now suppose that there exists $\mathcal{H}_1 \subset \mathcal{H}_2 \subset \mathcal{G}$, and consider the closed subspaces $L^2(\mathcal{H}_1)$, $L^2(\mathcal{H}_2)$ of square integrable random variables with respect to $\mathcal{H}_1$ and $\mathcal{H}_2$. It is clear that $L^2(\mathcal{H}_1) \subset L^2(\mathcal{H}_2)$, and by the previous comment $$P_{L^2(\mathcal{H}_1)}\cdot P_{L^2(\mathcal{H}_2)} = P_{L^2(\mathcal{H}_2)}\cdot P_{L^2(\mathcal{H}_1)}= P_{L^2(\mathcal{H}_1)}.$$
Now, we can interpert $P_{L^2(\mathcal{H}_1)}(X)$ as $\mathbb{E}\left[X|\mathcal{H}_1\right]$ and $P_{L^2(\mathcal{H}_2)}(X)$ as $\mathbb{E}\left[X|\mathcal{H}_2\right]$ to get the desired result.