Variance of a binomial where the number of draws is a random variable

binomial distributionrandom variables

Suppose I draw $M$ objects where $M \sim \mathrm{Poisson}(\lambda)$. Then for each of the $M$ objects, I decide whether or not to keep it by flipping a coin that lands on heads with probability $p$. At the end of this process, I have $N$ items. Clearly the expectation of $N$ is $\lambda p$. My question is, what is the variance of $N$? I know the variance of a binomial is $np(1-p)$, but I don't know how to proceed when $n$ is a random variable and not just a parameter. My intuition is that the variance will be higher than $\lambda p(1-p)$ but I'm not sure if that's right.

There is a similar question here What is the distribution of a binomial variable where the number of trials is itself random? but I can't quite follow the answers and variance is not discussed.

I'm also interested in the case where the distribution of $M$ is itself binomial.

Best Answer

First observe that

$(N|M=m)\sim Bin(m;p)$

To calculate $\mathbb{E}[N]=\lambda p$ you did

$$\mathbb{E}[N]=\mathbb{E}[\mathbb{E}(N|M=m)]=\mathbb{E}[mp]=p\mathbb{E}[M]=\lambda p$$

Using the fact that $\mathbb{V}(X)=\mathbb{E}[X^2]-\mathbb{E}^2[X]$ you can easily proceed in the same way

$$\mathbb{V}[N]=\mathbb{E}[N^2]-\mathbb{E}^2[N]=\mathbb{E}[\mathbb{E}(N^2|M=m)]-\lambda^2p^2=...=p(1-p)\lambda+\lambda p^2$$

.... Your intuition is right!

I'm also interested in the case where the distribution of M is itself binomial.

I hope the procedure I showed is clear... if yes you can use it in any case...

Edit: law of N

$$\mathbb{P}[N=k]=\sum_{n=k}^{\infty}\binom{n}{k}p^k(1-p)^{n-k}\frac{e^{-\lambda}\lambda^n}{n!}=$$

$$=\frac{(\lambda p)^k}{k!}e^{-\lambda}\sum_{(n-k)=0}^{\infty}\frac{[(1-p)\lambda]^{n-k}}{(n-k)!}= \frac{(\lambda p)^k}{k!}e^{-\lambda}e^{(1-p)\lambda}=\frac{(\lambda p)^k e^{-\lambda p}}{k!}$$

$k=0,1,2,..$

Related Solutions

Confusion on the proof and meaning of negative binomial random variables

There are at least four formulations of the negative binomial distribution if you consider in your word question:

Are you counting tosses or tails until you get $m$ heads? I.e. is the minimum possible value $m$ or $0$?
Does heads count as a failure (you go on until $m$ failures) or is tails a failure (you go on until $n$ successes? Which has probability $p$?

If you only count tails, then in a sense you are counting failures rather than attempts. Either way, there is no limit to the number of tails you may see before the $m$th head. So if tails are a failure then there is no limit to the possible number of failures.

Suppose, as you seem to be, you are counting tosses and the probability of a head is $p$. Then for the $k$th toss to be the $m$th head you need $m-1$ of the first $k-1$ tosses to be heads and the $k$th toss to be a head. This has probability ${k-1 \choose m-1}p^{m-1}(1-p)^{(k-1)-(m-1)} \times p$ which is $${k-1 \choose m-1}p^{m}(1-p)^{k - m}$$ and if you sum this from $k=m$ to $\infty$ you should get $1$

Wikipedia has a slightly different formulation but essentially the same result

Variance of square of binomial distribution

If you consider for $n=0,1,2,3$ so $2^n=1,2,4,8$, you get

n    2^n        E[p]               Var(p)
                                   
0     1           1                   0
1     2     0.5   = 1/2    0.25       = 1/2^2 
2     4     0.25  = 1/4    0.0935     = 3/2^5
3     8     0.125 = 1/8    0.02734375 = 7/2^8

which suggest a mean of $\dfrac{1}{2^n}$ and variance of $\dfrac{2^n-1}{2^{3n-1}}$

and if we let $m=2^n$ then they suggest a mean of $\dfrac{1}{m}$ and variance of $\dfrac{2}{m^2}-\dfrac{2}{m^3}$.

As for a proof, consider $q$ as a binomial random variable with parameters $m$ and $\frac12$:

$q$ has mean $\frac{m}{2}$ and variance $\frac{m}{4}$ and excess kurtosis $-\frac{2}{m}$,

so $\mathbb E\left[\frac2m q-1\right]=0$ and thus, with $p=\left(\frac2m q-1\right)^2$, you can say

$\mathbb E[p]=\mathbb E\left[\left(\frac2m q-1\right)^2\right]=\text{Var}\left(\frac2m q-1\right)=\text{Var}\left(\frac2m q\right)=\left(\frac2m\right)^2\frac{m}{4}=\frac{1}{m},$
$\mathbb E[p^2]=\mathbb E\left[\left(\frac2m q-1\right)^4\right]=\left(\frac2m\right)^4\left(3-\frac{2}{m}\right)\left(\frac{m}{4}\right)^2=\frac{3}{m^2}-\frac2{m^3},$
$\text{Var}(p)= \left(\frac{3}{m^2}-\frac2{m^3}\right) -\left(\frac1m\right)^2 = \frac{2}{m^2}-\frac2{m^3}.\qquad \square$

Best Answer

Related Solutions

Confusion on the proof and meaning of negative binomial random variables

Variance of square of binomial distribution

Related Question