Is the variance of the mean of a set of possibly dependent random variables less than or equal to the average of their respective variances?
Mathematically, given random variables $X_1, X_2, …, X_n$ that may be dependent:
Let $\bar{X} = \frac{1}{n}\sum_{i=1}^n X_i$ be the mean of these random variables.
Is it true that:
$$\text{Var}(\bar{X}) \leq \frac{1}{n}\sum_{i=1}^n \text{Var}(X_i)$$
I know that for independent random variables, we have the following equality:
$$\text{Var}(\bar{X}) = \frac{1}{n^2}\sum_{i=1}^n \text{Var}(X_i)$$
Which clearly satisfies the inequality. However, I'm unsure if this holds for dependent variables.
If this inequality is true, is there a proof or intuitive explanation?
If it's not always true, are there conditions under which it holds? What about the following inequality?
$$\text{Var}(\bar{X}) \leq \text{Max}_{i=1}^n \text{Var}(X_i)$$
Any insights, proofs, or counterexamples would be greatly appreciated. Thank you!
Best Answer
Yes, it is true. Here is a proof. $$ \begin{align} \newcommand{\Var}{\operatorname{Var}} &\Var(\overline{X}) \\ &= \frac1{n^2}\Var\left(\sum_{i=1}^n X_i\right) \\ &=\frac1{n^2}\sum_{i=1}^n\sum_{j=1}^n\text{Cov}(X_i,X_j) \\ &\le\frac1{n^2}\sum_{i=1}^n\sum_{j=1}^n\sqrt{\text{Var}(X_i)\cdot \Var(X_j)} \\ &\le\frac1{n^2}\sum_{i=1}^n\sum_{j=1}^n\frac{\Var X_i+\Var X_j}{2} \\ &= \frac1n\sum_{i=1}^n \Var(X_i). \end{align} $$