[Math] Covariance between given iid random variable and sample average

covariancemeansvariance

Apologies if this may look like a silly question.

Consider a sample of size $N$. Let $y_i$ be the $i$-th realization of a random variable that is independently and identically distributed with mean $\mu$ and variance $\sigma^2$.

Let $\bar{y}$ be the mean of $y_i$ across all $i$. We know that $Var(\bar{y})=\sigma^2/N$.

How about $Cov(y_i,\bar{y})$? I would say that

$Cov(y_i,\bar{y})=Cov\left(y_i,\frac{\sum_i y_i}{N}\right)=\frac{1}{N}\left[Var(y_i)+(N-1)Cov(y_i,y_j)\right]=\sigma^2/N$

Where the latter follows because of the independence assumption. Is that right? Many thanks.

Best Answer

Overall this looks good. I have a few minor concerns and comments:

  • It would help to specify the definition $\overline{Y}= \frac{1}{N}\sum_{i=1}^N Y_i$. (At first I was confused since I thought you meant $\overline{Y}=E[Y_i]$).

  • I would personally prefer to write your equality as $$ Cov(Y_i, \overline{Y}) = \frac{1}{N}\left[Var(Y_i) + \sum_{j \neq i} Cov(Y_i, Y_j)\right] $$ This avoids the "dangling and undefined index $j$" in
    $$ Cov(Y_i, \overline{Y}) = \frac{1}{N}\left[Var(Y_i) + (N-1)\underbrace{Cov(Y_i,Y_j)}_{\mbox{undefined $j$}}\right]$$

  • The basic principles you are applying are \begin{align} Cov(Z, aW) &= a Cov(Z, W)\\ Cov\left(Z, \sum_{i=1}^N R_i\right) &= \sum_{i=1}^N Cov(Z, R_i) \end{align} for any random variables $Z, W, R_1, ..., R_N$ with finite means and variances, and any scalar $a$ and positive integer $N$.

Related Question