Suppose you have two random variables
$X:\Omega \to \mathbb{R}$ and $Y:\Omega \to \mathbb{R}$
which are not necessarily independent.
How is the product $XY$ defined and how do I calculate the image of the new random variable $Z:=XY$?
In the context of covariance of finite random variables we are told to simply multiply all elements of each image, $X(\Omega)$ and $Y(\Omega)$, with each other. However, we are not provided with any formal definition. So I have no idea what to do when we have non-finite random variables.
Which makes it even more confusing is that the professor uses the following notation when he states a Lemma about the expected value of $XY$ when both random variables are independent:
Lemma:
The expected value, $\mathbb{E}(XY)$ exists iff $\sum\limits_{\omega\in \Omega}|X(\omega)Y(\omega)|p(\omega)<\infty$.
Proof
$\sum\limits_{\omega\in \Omega}|X(\omega)Y(\omega)|p(\omega)=\sum\limits_{x\in X(\Omega)}\sum\limits_{y\in Y(\Omega)}|xy|\sum\limits_{\omega: X(\omega)=x, Y(\omega)=y}p(\omega)= $ …
So here it seems that the expression $XY$ is treated like the multiplication of two real-valued functions.
Can someone give me an explanation of $XY$?
Best Answer
First to address an apparent point of confusion in the comments: "$X(\Omega)=\{X(\omega_1)=x_1,X(\omega_2)=x_2\}$ and $Y(Ω)=\{Y(\omega_1)=y_1,Y(\omega_2)=y_2\}$."
That is not how random variables work in general. In general, the preimage of $x_1$ under $X$ is some subset of $\Omega,$ not just a single value $\omega_1,$ and likewise for the preimage of $y_1$ under $Y.$ It is possible for these two preimages to overlap but to be different from each other.
For example, we might have $\Omega = \{\omega_1, \omega_2, \omega_3, \omega_4\}$ and furthermore \begin{align} X(\omega_1) &= X(\omega_2) = x_1, \\ X(\omega_3) &= X(\omega_4) = x_2, \\ Y(\omega_1) &= Y(\omega_3) = y_1, \\ Y(\omega_2) &= Y(\omega_4) = y_2. \end{align}
In this way $p(\omega_1) = \mathbb P(X=x_1,Y=y_1),$ $p(\omega_2) = \mathbb P(X=x_1,Y=y_2),$ $p(\omega_3) = \mathbb P(X=x_2,Y=y_1),$ and $p(\omega_4) = \mathbb P(X=x_2,Y=y_2).$ Similarly, $X(\omega_1)Y(\omega_1)p(\omega_1) = x_1 y_1 \mathbb P(X=x_1,Y=y_1)$ and so forth.
So now if $P(\omega_1) = P(\omega_2) = P(\omega_3) = P(\omega_4) = \frac14,$ then $X$ and $Y$ would be independent variables. But if $P(\omega_1) = P(\omega_4) = \frac12$ and $P(\omega_2) = P(\omega_3) = 0$ with $x_1\neq x_2$ and $y_1\neq y_2$ then $X$ and $Y$ can be expressed as functions of each other.
Consider the case where $P(\omega_1) = P(\omega_4) = \frac13$ and $P(\omega_2) = P(\omega_3) = \frac16,$ and suppose $x_1= y_1=1$ and $x_2= y_2=2.$ Then \begin{align} x_1y_1P(x_1,y_1) &= 1\cdot 1\cdot\frac13 = \frac13,\\ x_1y_2P(x_1,y_2) &= 1\cdot 2\cdot\frac16 = \frac13,\\ x_2y_1P(x_2,y_1) &= 2\cdot 1\cdot\frac16 = \frac13,\\ x_2y_2P(x_2,y_2) &= 2\cdot 2\cdot\frac13 = \frac43,\\ \mathbb{E}(X) &= \mathbb{E}(Y) = \frac32, \end{align} and therefore \begin{align} \mathrm{cov}(XY) &= \left(\sum_{\omega\in\Omega} X(\omega)Y(\omega)P(\omega)\right) −\mathbb{E}(X)\mathbb{E}(Y) \\ &= x_1y_1P(x_1,y_1)+x_1y_2P(x_1,y_2)+x_2y_1P(x_2,y_1)+x_2y_2P(x_2,y_2) −\mathbb{E}(X)\mathbb{E}(Y) \\ &=\frac73 - \frac94 = \frac1{12}. \end{align}
All the definitions and formulas work perfectly well. There is no inconsistency among them.
In general, if $Z = XY$ where $X$ and $Y$ are random variables, then $Z(\omega) = X(\omega) Y(\omega)$ for any $\omega \in \Omega.$ In other words, it is the product of two functions on the probability space. This is true whether the probability space has a finite or infinite number of elements.
The summation notation $\sum_{\omega\in\Omega}$ is good for finite probability space or for a countably infinite probability space. For continuous distributions one generally uses some form of integration rather than summation. But in any case the interpretation of $Z=XY$ as $Z(\omega)=X(\omega)Y(\omega)$ is true for every $\omega$ in the probability space no matter how many elements the space has.