Square-Loss Minimizer – Conditional Expectation Over Continuous Functions

Question

It is well-known that the conditional expectation of a square-integrable random variable $Y$ given another (real) random variable $X$ can be obtained by minimizing the mean square loss between $Y$ and $f(X)$ for all Borel-measurable functions $f : \mathbb{R} \rightarrow \mathbb{R}$, that is:

$$\tag{1}\mathbb{E}[Y \,|\,X] \, = \, \operatorname{argmin}_{f\in\mathcal{B}}\mathbb{E}|Y – f(X)|^2$$

for $\mathcal{B}:=\{g : \mathbb{R} \rightarrow \mathbb{R} \mid g \ \text{ Borel measurable}\}$.

Question: Are you aware of conditions on $(X,Y)$ for which in fact

$$\tag{2}\mathbb{E}[Y \,|\, X] \, = \, \operatorname{argmin}_{f\in\mathcal{C}}\mathbb{E}|Y – f(X)|^2 \quad \text{ with } \quad \mathcal{C}:= \{g : \mathbb{R}\rightarrow\mathbb{R} \mid g \ \text{ continuous} \} \ ?$$

Answer 1

$\newcommand\R{\mathbb R}\newcommand\B{\mathscr B}$The question can be restated as follows: When does there exist a continuous function $g\colon\R\to\R$ such that $E(Y|X)=g(X)$ almost surely (a.s.)?

A sufficient condition for this is as follows. Let $\B(\R)$ denote the Borel $\sigma$-algebra over $\R$. Let $\R\times\B(\R)\ni(x,B)\mapsto\nu_x(B)\in\R$ be a regular conditional distribution of $Y$ given $X$ (which exists), so that

• the function $\R\ni x\mapsto\nu_x(B)\in\R$ is Borel-measurable for each $B\in\B(\R)$,
• the function $\B(\R)\ni B\mapsto\nu_x(B)\in\R$ is a probability measure for each $x\in\R$,
• $P(X\in A,Y\in B)=\int_A P(X\in dx)\nu_x(B)$ for each $(A,B)\in\B(\R)\times\B(\R)$.

Suppose also that

• the function $\R\ni x\mapsto\nu_x$ is continuous wrt the topology of weak convergence of probability measures and
• for each $x\in\R$ there is some real $r>0$ such that the identity function $\R\ni y\mapsto y\in\R$ is uniformly integrable wrt to the set $\{\nu_z\colon|z-x|<r\}$ of measures.

Then the function $\R\ni x\mapsto g(x):=\int_\R y\nu_x(dy)$ is continuous and $E(Y|X)=g(X)$ a.s.

---

A special case of the above sufficient condition is as follows. Suppose that the (joint) distribution of $(X,Y)$ is absolutely continuous wrt the Lebesgue measure on $\B(\R^2)$ with a joint pdf $f_{X,Y}$. Let $f_X$ be the pdf of $X$. Suppose that there is a nonnegative Borel-measurable function $f_{Y|X}\colon\R^2\to\R$ such that

• $f_{Y|X}(x,y)$ is continuous in $x\in\R$ for each $y\in\R$,
• $f_{X,Y}(x,y)=f_{Y|X}(x,y)f_X(x)$ for all $(x,y)\in\R^2$, and
• for each $x\in\R$ there is some real $r>0$ such that the identity function $\R\ni y\mapsto y\in\R$ is uniformly integrable wrt to the set $\{\nu_z\colon|z-x|<r\}$ of measures, where $\nu_z(dy):=f_{Y|X}(x,y)\,dy$.

Then the function $\R\ni x\mapsto g(x):=\int_\R y f_{Y|X}(x,y)\,dy$ is continuous and $E(Y|X)=g(X)$ a.s.