I am unable to understand the practical use of Covariance and Variance.
In my understanding, Covariance and Correlation are both measures of how one variable changes with respect to another. The only difference I see is that Correlation is scaled down to [-1, 1]
Similarly Variance and Standard Deviation of how spread out a distribution is. The main difference here is that Standard Deviation is scaled down (square root of variance).
I understand the individual differences between Covariance/Correlation and Variance/Standard Deviation. The reason I have grouped these 4 together in this question is the common phrase scaled down. I am learning computer vision programming and I was wondering why would I ever use something which is not bounded (like covariance). Similarly, Standard Deviation will give a smaller range than variance to process data.
Best Answer
When you have a linear combination of independent random variables $z = \sum w_i.x_i$, and you want to get the variance of $z$, you can just calculate a weighted sum of variances $\sigma_z = \sum w_i^2 . \sigma_i$. With standard deviation, it is a slightly more complicated formula. So in some sense, variance is simpler to work with, because it is additive for independent random variables.
Before I explain the covariance part of the question, I want to make sure you understand linear algebra notation. Another way of writing $z = \sum w_i.x_i$ is $z = w^Tx$.
When you want to calculate the variance of a linear combination of non-independent variables, you simply need to calculate $\sigma_z = w^T \Sigma w$, where $\Sigma$ is the covariance matrix. You can do this using correlation but it is a more complicated formula. So in a similar sense as before, covariance is simpler to work with than correlation.
The advantages of correlation and standard deviation are that they seem to be more intuitive for most people. But when you are writing code, they are typically cumbersome to work with.