From this lecture notes (Page 7, Example 4), the space of univariate random variables with finite variance is a Hilbert space, when the inner product is chosen as $\langle X, Y \rangle = \mathrm{Cov} (X, Y)$. I'm asking for answers or references for the following two questions:
(1) How to generalize the prescribed idea of constructing Hilbert space to multi-variate random variables?
(2) Can we define a Hilbert space of random variables, with inner product as $\langle X, Y \rangle = \mathbb{E} (X^T Y)$?
Best Answer
For a multivariate random variable $X = (X_1,\ldots X_d)$, we normally define the expected value of $X$ to be defined to be $\mathbb{E}(X) = (\mathbb{E}(X_1),\ldots \mathbb{E}(X_d))$ and say that a multivariate random variable $X$ has finite variance if $\mathbb{E}[|X - \mathbb{E}[X]|^2]<\infty$ where $|\cdot|$ is the standard $l^2$ norm on $\mathbb{R}^d$. You can then extend your inner product for $d=1$ to be given by $\left< X,Y\right> = \mathbb{E}[(X-\mathbb{E}[X])\cdot (Y-\mathbb{E}[Y])]$, where the $\cdot$ is represents your traditional dot product. This gives the same definition for your definition in $d=1$ but is defined for arbitrary $d$. This function is in fact an inner product and inherits its properties from the expected value operator. This said, I am not sure if there are other inner products on multivariate random variables to reduce to your case if $d=1$.
That's exactly how the Hilbert space is defined normally for multivariate random variables.