[Math] Variance of a sum of dependent random variables

covarianceprobabilityrandom variables

Let's assume we have a random variable $z$ which really is a sum of two correlated variables:

$$z \sim x + \operatorname{Bin}(x, p),$$

where $p$ is a constant. $x \sim \operatorname{Poiss}(\lambda)$.

What I'm trying to do here is to find variance of $z$. Probably you wanna suggest convergence $$\lim_{n \rightarrow ∞} \operatorname{Bin}(n, p) = \operatorname{Poiss}(np)$$

but unfortunately I can't use it — p is too big, n is too small and I wanna get exact value of the variance.

As you know, $\operatorname{V}[x + y] = \operatorname{V}[x] + \operatorname{V}[y] + 2 \operatorname{Cov}(x, y)$.

The point is, I don't know how to find $\operatorname{Cov}(x, \operatorname{Bin}(x, p))$.

Is there any way to evaluate it analytically?

Best Answer

$$\text{cov}\,(X,Y) = \newcommand{\E}{\Bbb E} \E XY - \E X \E Y$$

By the Tower Law of Conditional Expectation, if $Y|X\sim \text{Bin}(X,p)$, $$\E [X Y] = \E X \E[Y|X], \quad \E Y=\E \E[ Y|X] $$ Of course, $\E[Y|X]=pX$, $\E X = \lambda$, $\E X^2 = \lambda^2+\lambda$. Plugging in gives

$$\text{cov}\,(X,Y) = \newcommand{\E}{\Bbb E} p\E (X^2) - \lambda p\E X = p\left[\lambda^2+\lambda-\lambda^2\right] = p\lambda$$


(note that this is consistent with the intuition that if $Y$ depends on $X$, then $\text{cov}(X,Y) \geq 0$.)

To finish what you wanted to do, we need to calculate $\Bbb VY = \E Y^2 - (\E Y)^2$. We used already that $\E Y = p\lambda$. We compute:

$$ \E Y^2 = \E \E[Y^2 | X] = \E[Xp(1-p) + X^2p^2] = p\lambda[(1-p)+p(\lambda+1)] = p\lambda (1+p\lambda)$$

so that $\Bbb V Y = p\lambda$ and therefore, $$ \Bbb V(Z) = \lambda + p\lambda + 2p\lambda = \lambda + 3p\lambda$$

Related Question