I was messing around with my grandfather's old math textbooks and came across this problem:
Suppose $\Omega = \left\{\omega_{1}, \omega_{2}, \ldots, \omega_{n}\right\}$ a discrete space and $p = \left(p_{1}, p_{2}, \ldots, p_{n}\right)$ a discrete distribution in $\Omega$. Now, let $X, Y:\Omega\to\mathbb{R}$ be two random variables, which we can understand like vectors
$$ X = \left(x_1, x_2, \ldots, x_n\right), $$
$$ Y = \left(y_1, y_2, \ldots, y_n\right). $$
Show that the Cauchy-Schwarz inequality states that
$$ \left| Cov\left(X, Y\right)\right|\leq\sqrt{Var\left[X\right]}\sqrt{Var\left[Y\right]}. $$
My original thought was to define the mean as the dot product, but it doesn't make any sense since we have the discrete distribution $p$. Then I thought to define the dot product as
$$X\cdot Y = \sum_{i=1}^{n}{p_i x_i y_i} = \mathbb{E}[XY]$$
and continue on proving the Cauchy – Schwarz inequality using known probability theory theorems for the mean and so on. So am I completely wrong? Any help will be appreciated.
Best Answer
Covariance satisfies the properties required to be an inner product: it is linear in both of it's arguments as well as non-negative. Taking $\langle\cdot, \cdot\rangle$ to mean $Cov(\cdot, \cdot)$, then by the Cauchy Schwarz inequality:
$$|\langle X, Y\rangle| \leq \sqrt{\langle X,X\rangle} \sqrt{\langle Y,Y\rangle}$$
which is by definition the inequality you're trying to prove.