Consider a random variable $X \colon\Omega \rightarrow \mathbb{R}$ for a probability space $(\Omega, \mathcal{F}, P)$.
We had the following definition for the expectation:
$$\mathbb{E}[X]= \int_{\Omega} X(\omega) d P(\omega). \quad (1)$$
For the discrete case we would probably write
$$\mathbb{E}[X]= \sum_{ \omega \in \Omega} X(\omega) P(\omega). \quad (2)$$
Now as non-mathematician (i.e. not having much knowledge about integrals and measure theory),
I am wondering why converting the sum in (2) to an integral does not result in
$$\mathbb{E}[X]=\int_{\Omega} X(\omega) P(\omega) d \omega\quad (3)$$
instead of (1).
(3) is what I would expect, because instead of the sum we just integrate, hence informally "replace sum in (2) by integral sign and add a $d \omega$ at the end".
What is the difference between (1) and (3)? Is it valid to write (3)? Is it just "notation" for measures $P$ or is there a reason for this?
Thank you very much and please have patient with a non-mathematician that never had a lecture in mass theory or integrals.
Best Answer
One writes $${\Bbb E}[X]=\int_\Omega X(\omega)\>{\rm d}P(\omega)$$ if only the probability measure $P:\>{\cal F}\to[0,1]$ is at stake, which assigns values to subsets $A\subset\Omega$, not to individual points $\omega\in\Omega$.
But often the probability space $\Omega$ has a "natural geometric measure", say length or area. This is particularly the case if $\Omega$ is a reasonable subset of some ${\mathbb R}^n$. Integration with respect to this geometric measure is then signalled by writing ${\rm d}\omega$. This has as yet nothing to do with probability. The latter is an additional feature which makes its appearance as a weight factor $\omega\mapsto f(\omega)$, called the probability density on $\Omega$. Such an $f$, if it exists, is completely determined by the given probablity measure $P$, and is related to $P$ via the formula $$P[A]=\int_A f(\omega)\>{\rm d}\omega\ .$$ There is no standard notation for this density; lets just call it $f$ for the moment. The standard normal $f(x):={1\over\sqrt{2\pi}}e^{-x^2/2}$ on ${\mathbb R}$ is an example. Given such a density the expectation of a random variable $X:\>\Omega\to{\mathbb R}$ is then given by $${\Bbb E}[X]=\int_\Omega X(\omega)\>f(\omega)\>{\rm d}\omega\ .$$