I am currently working on my understanding of regression fundamentals and I checked this source (one can find the (even exact) same statement in multiple sources).
In Theorem 3.1.1, the author claims to prove the so called Conditional Expectation Function (CEF) Decomposition property, which states:
Theorem 3.1.1:
We have
$$Y_i=E(Y_i|X_i)+\epsilon_i$$
with the property that $(a)$ $E(\epsilon_i|X_i)=0$ and $(b)$ $E(f(X_i)\epsilon_i)=0$ for any function $f$.
The problem is, the proof only checks $(a)$ and $(b)$ but never checks the actual existence of the decomposition of $Y_i=E(Y_i|X_i)+\epsilon_i$.
The author then claims that
This theorem says that any random variable,
$Y_i$
, can be decomposed into a piece that's "explained by
$X_i$", i.e., the CEF, and a piece left over which is orthogonal to (i.e., uncorrelated with) any function of $X_i$
.
Do we have to prove this or comes this as some sort of assumption (couldn't find it, but surely one could just build the model like this) or is this a result from somewhere else, like factorization lemma or so?
The whole thing sounds a bit sloppy, clearly things like measurability are dropped, but still, what do I miss?
Best Answer
This follows from the equality $$ Y_i=\mathbb E\left[Y_i\mid X_i\right]+Y_i-\mathbb E\left[Y_i\mid X_i\right]; $$ defining $\varepsilon_i:=Y_i-\mathbb E\left[Y_i\mid X_i\right]$ gives that $\mathbb E\left[\varepsilon_i\mid X_i\right]=0$, from which $\mathbb E\left[f\left(X_i\right)\varepsilon_i\right]=0$ follows.