I'm doing some basic error, but I just can't see where…
Let $X_i\sim \mathrm{Bin}(\theta_i,n)$, and $X_j\sim \mathrm{Bin}(\theta_j,n)$
I want to find $\mathrm{Cov}(X_i,X_j)$.
So,
$E(X_iX_j)=E(E(X_iX_j|X_i=x_i))=E(X_i n \theta_j)=n\theta_i n\theta_j$. But this isn't correct is it? Why?
Best Answer
Recall that $X=(X_i)_i$ multinomial $(n,(\theta_i)_i)$ with $\sum\limits_i\theta_i=1$ can be realized as $$X_i=\sum\limits_{k=1}^n\mathbf 1_{U_k=i},$$ for every $i$, where $(U_k)$ is i.i.d. with $P(U_k=i)=\theta_i$ for every $k$ and every $i$. Thus, for every $i\ne j$, $$ X_iX_j=\sum_k\mathbf 1_{U_k=i}\mathbf 1_{U_k=j}+\sum_{k\ne\ell}\mathbf 1_{U_k=i}\mathbf 1_{U_\ell=j}.$$ Since $[U_k=i]\cap[U_k=j]=\varnothing$ for every $k$ and the events $[U_k=i]$ and $[U_\ell=j]$ are independent for every $k\ne\ell$, this yields $$ E(X_iX_j)=\sum_{k\ne\ell}P(U_k=i)P(U_\ell=j)=n(n-1)\theta_i\theta_j. $$ Since $E(X_i)=n\theta_i$ and $E(X_j)=n\theta_j$, one gets $$ \mathrm{Cov}(X_i,X_j)=n(n-1)\theta_i\theta_j-n\theta_in\theta_j=-n\theta_i\theta_j.$$