$\DeclareMathOperator{E}{\mathbf{E}}$
Let $X$ and $Y$ be $\mathbb{R}^n$ and $\mathbb{R}$-valued random variables, respectively. Let $\mu: \mathbb{R}^n \to \mathbb{R}$ be a regression function, i.e., a function such that $\mu(X)$ is (a version of) the conditional expectation $\E(Y \mid X)$.
Assume that:
- $\mu$ is continuous; and
- $X$ has a density that is positive everywhere.
Is it true that for all $x \in \mathbb{R}^n$,
$$
\mu(x) = \lim_{\epsilon \to 0} \E(Y \mid \|X – x\| \leq \epsilon)?
$$
If this is indeed true, and the density of $f$ is bounded, is it also true that the limit above converges uniformly in $x$?
I know the measure theoretic definition of the conditional expectation, and I take it as a definition that when $F$ is an event with positive probability, $\E(Y \mid F) = \E(Y \mathbf{1}_F) / \Pr(F)$. However, I can't quite get a handle on how to use these definitions to prove or disprove the above statement, even though it seems intuitive.
Best Answer
First observe that since the event $\{ \|X - x\| \leq \epsilon\}$ is $\sigma(X)$-measurable, $$ \mathbb E(Y \mid \|X - x\| \leq \epsilon)=\frac 1{\mathbb P\left(\|X - x\| \leq \epsilon\right)}\mathbb E(Y \mathbf{1}\{ \|X - x\| \leq \epsilon\})=\frac 1{\mathbb P\left(\|X - x\| \leq \epsilon\right)}\mathbb E(\mu(X)\mathbf{1}\{ \|X - x\| \leq \epsilon\}).$$ Therefore, $$ \lvert \mu(x)-\mathbb E(Y \mid \|X - x\| \leq \epsilon)\rvert=\left\lvert \frac 1{\mathbb P\left(\|X - x\| \leq \epsilon\right)}\mathbb E(\left(\mu(x)-\mu(X)\right)\mathbf{1}\{ \|X - x\| \leq \epsilon\})\right\rvert \leqslant \sup_{y:\|y - x\| \leq \epsilon}\lvert \mu(x)-\mu(y)\rvert$$ and we conclude by continuity of $\mu$ at $x$.
If $\mu$ is uniformly continuous, then the convergence is uniform in $x$.