[Math] Expectation with respect to empirical distribution

measure-theoryprobabilityprobability theory

Let $(\Omega,\mathcal{A})$ be a measure space and $X$ a random variable with distribution $P$. The expectation of some measurable function $g$ with respect to $P$ is

$$
\mathbb{E}_P[g(X)] = \int_\Omega g(X(\omega))\, dP(\omega).
$$

The empirical distribution $Q$ of $P$ is defined as

$$
Q(B) = \frac{1}{n} \sum_{i=1}^n \mathbb{1}_B(x_i) \quad B \in \mathcal{A},
$$
where $x_1,\dotsc,x_n$ are i.i.d. samples from $P$.

I know that
$$
\mathbb{E}_Q[g(X)] = \frac{1}{n} \sum_{i=1}^n g(x_i)
$$
but it is not obvoius how to derive this starting with
$$
\mathbb{E}_Q[g(X)] = \int_\Omega g(X(\omega))\, d\left[ \frac{1}{n} \sum_{i=1}^n \mathbb{1}_\omega(x_i) \right]
$$

Best Answer

I think for any realization of random variable : $ X_{1},...,X_{n}$, the empirical distribution $F_{n}(x)=\frac{1}{n}\sum_{i=1}^{n}I_{\{X_{i}\le x\}}$ is just a discrete distribution function concentrated on these n value and attached weight $\frac{1}{n}$ to each of them. Thus, the integral $\int g(x)dF_{n}(x)$ is same as expected value of $g(x)$ with respect to a discrete distribution: $$\int g(x)dF_{n}(x)=\frac{1}{n}\sum_{i=1}^{n}g(X_{i})$$ I hope this is what you are looking for.

Related Question