Let $(\Omega, \mathscr{F}, \mathscr{F}_k, P)$ be a filtered probability space and $(X_k, \mathscr{F}_k)$ be a Markov chain. I would like to show that the Markov property $$E[f(X_{k+1})|\mathscr{F}_k] = E[f(X_{k+1})|X_k]$$ for all bounded measurable $f$ is equivalent to the following : for every $k$, bounded $\sigma(X_j, j\ge k)$-measurable random variable $Y$ and bounded $\mathscr{F}_k$ -measurable random variable $Z$,
$$E[YZ|X_k] = E[Y|X_k]E[Z|X_k].$$
The proof showing that the latter implies the former goes as follows: If $Z$ is bounded and $\mathscr{F}_k$-measurable we obtain
$E[f(X_{k+1})Z]=E[E[f(X_{k+1})Z|X_k]]=E[E[f(X_{k+1})|X_k]E[Z|X_k]]=E[E[f(X_{k+1})|X_k]Z]$.
I don't know how we get the last equality here. What property of conditional expectation gives this identity?
Best Answer
We show that if $W$ is $X_k$ measurable, than for every $Z$:
$E[WZ]=E[WE[Z|X_k]]$ [1]
Indeed this is an application of the tower property:
$E[WZ]=E[E[WZ]|X_k]=E[WE[Z|X_k]]$
Now choose in [1] $W=E[f(X_{k+1})|X_k]$ and you should have your last equality.
Admittedly, the amount of E[...] in this expressions does not really help to follow the algebra...