Probability – How to Estimate p for a Binomial Distribution and Understand Its Variance

binomial distributionprobabilityvariance

how can I calculate the variance of p as derived from a binomial distribution? Let's say I flip n coins and get k heads. I can estimate p as k/n, but how can I calculated the variance in that estimate?

I'm interested in this so that I can control for variance in my ratio estimates when I'm comparing between points with different numbers of trials. I'm more sure of the estimate of p when n is greater, so I would like to be able to model how reliable the estimate is.

Thanks in advance!

example:

  • 40/100. The MLE of p would be 0.4, but what is the variance in p?
  • 4/10. The MLE would still be 0.4, but the estimate is less reliable, so there should be more variance in p.

Best Answer

If $X$ is $\text{Binomial}(n, p)$ then MLE of $p$ is $\hat{p} = X/n$.

A binomial variable can be thought of as the sum of $n$ Bernoulli random variables. $X = \sum_{i=1}^n Y_i$ where $Y_i\sim\text{Bernoulli}(p)$.

so we can calculate the variance of the MLE $\hat{p}$ as

$$\begin{align*} \text{Var}[\hat{p}] &= \text{Var}\left[\dfrac{1}{n}\sum_{i=1}^n Y_i\right]\\ &= \dfrac{1}{n^2}\sum_{i=1}^n Var[Y_i]\\ &= \dfrac{1}{n^2}\sum_{i=1}^n p(1-p)\\ &= \dfrac{p(1-p)}{n} \end{align*}$$

So you can see that the variance of the MLE gets smaller for large $n$, and also it is smaller for $p$ close to 0 or 1. In terms of $p$ it is maximized when $p=0.5$.

For some confidence intervals you can check out Binomial Confidence Intervals

Related Question