Probability Distributions – Sum of Differently Weighted Coins

probabilityprobability distributions

Imagine that we have $100$ coins, which each have a random weight between $0$ and $1$ $\left(~\mbox{but we know the weights}~\right)$:

  • That is to say, we have $100$ known numbers $p_{1}, p_{2},\ldots, p_{100} \in \left[0, 1\right]$ such that
    • coin $n$ has probability $p_{n}$ of flipping heads, which we treat as
      an outcome of $1$,
    • and probability $1 – p_{n}$ of flipping tails,
      which we treat as an outcome of $0$.
  • We want the probability distribution of the sum of these $100$ coins. How can we compute it efficiently, exactly if possible or approximately if not possible $?$.

Best Answer

Since $100$ is a small enough number, you can compute the entire distribution exactly with the help of a computer.

More generally, say that there are $m$ coins total, with probabilities $p_1,p_2,\dots,p_m$ of heads, and you want the full probability distribution for the total number of heads. For each $k\in \{0,1,\dots,m\}$, let $q^m_k$ be the probability that there are exactly $k$ heads among the $m$ coins. You can prove that $$ q^m_k=p_m\cdot q^{m-1}_{k-1}+(1-p_m)\cdot q^{m-1}_k\tag1 $$ This follows by conditioning on the outcome of the $m^\text{th}$ coin. Either the $m^\text{th}$ coin is heads, and you need exactly $k-1$ heads among the first $m-1$ coins, or the $m^\text{th}$ coin is tails, and you need exactly $k$ heads among the first $m-1$ coins.

Using $(1)$, you can compute $q^m_k$ for all $k\in \{0,1,\dots,m\}$ by filling out an $(m+1)\times (m+1)$ table, where the rows and columns are labeled from $0$ to $m$. The entry in the $i^\text{th}$ row and $j^\text{th}$ column should be filled with $q^i_j$, using formula $(1)$, together with the numbers in the previous row. You need to fill the rows in the order $0,1,\dots,m$, so the numbers in the previous row are available when you need them. The zeroth row is the easiest; $q^0_0=1$, while $q^0_j=0$ for all $j>0$. This takes $O(m^2)$ additions and multiplications, which is manageable by a computer when $m=100$.

There do exists faster methods. Consider a small case, say $m=4$ and $k=3$. You can show that $$ q^4_3=p_1p_2p_3{(1-p_4)}+p_1p_2(1-p_3)p_4+p_1(1-p_2)p_3p_4+(1-p_1)p_2p_3p_4. $$ Now, define $r_i=p_i/(1-p_i)$ for each $i\in \{1,2,3,4\}$. Then $$ q^4_3=(1-p_1)(1-p_2)(1-p_3)(1-p_4)[r_1r_2r_3+r_1r_2r_4+r_1r_3r_4+r_2r_3r_4] $$ The expression $r_1r_2r_3+r_1r_2r_4+r_1r_3r_4+r_2r_3r_4$ is known as "the third elementary symmetric polynomial in four variables", and is usually notated as $e_3(r_1,r_2,r_3,r_4)$. In general, as long as you can compute $e_k(x_1,\dots,x_n)$, the $k^\text{th}$ elementary symmetric polynomial in $n$ variables, then you can compute $q^n_k$. For a fast method to compute $e_k(x_1,\dots,x_n)$, see this Computer science stack exchange question.

Related Question