Suppose that $n$ independent trials – each of which results in either outcome $1, 2, \dots, r$ with respective probabilities $p_1, p_2, …, p_r$ such that $\sum_i p_i = 1$. Let $N_i$ denote the number of trials when the outcome was $i$ then:
- Find the joint probability mass function of $N_1,\dots ,N_r$. This is called the multinomial distribution.
- Compute covariance of $N_i$ and $N_j$
- Compute mean and variance of the number of outcomes that do not occur.
The first two parts are pretty easy, I used indicator variables and some properties of independence to prove them. But I cannot make sense of what the third part is asking. I need some help in understanding the problem.
Best Answer
Here is my solution:
Let $I_i = \mathbb{I}(N_i = 0)$ be indicator that the outcome $i$ does not occur.
$M = \sum_{i=1}^r I_i$ - number of outcomes we are interested in.
$ \mathbb{E} M = \mathbb{E}(\sum_{i=1}^r I_i) = \sum_{i=1}^r \mathbb{E}I_i = \sum_{i=1}^r P(N_i = 0) = \sum_{i=1}^r (1 - p_i)^n $,
where $P(N_i = 0) = (1 - p_i)^n$ can be seen as binomial distribution with probability $p_i$.
$Var(M) = Var(\sum_{i=1}^r I_i) = \sum_{i=1}^r Var(I_i) + \sum_{i,j=1 \ i \neq j}^r Cov(I_i, I_j)$
a) $Var(I_i) = \mathbb{E} (I_i^2) - (\mathbb{E} I_i)^2 = \mathbb{E} I_i - (\mathbb{E} I_i)^2 = \mathbb{E} I_i (1 - \mathbb{E} I_i) = P(N_i = 0) (1 - P(N_i = 0)) = (1 - p_i)^n (1 - (1 - p_i)^n) $
b) $Cov(I_i, I_j) = \mathbb{E} (I_i I_j) - (\mathbb{E} I_i) (\mathbb{E} I_j) $
$\mathbb{E} (I_i I_j) = P(N_i = 0, N_j = 0) = (1 - p_i - p_j)^n$, as we can once again see it as a binomial distribution of the event "outcome $i$ or outcome $j$ occurs", which has probability $(p_i + p_j)$.
Thus:
$Cov(I_i, I_j) = (1 - p_i - p_j)^n - (1 - p_i)^n (1 - p_j)^n $,
And:
$Var(M) = \sum_{i=1}^r (1 - p_i)^n (1 - (1 - p_i)^n) + \sum_{i,j=1 \ i \neq j}^r ((1 - p_i - p_j)^n - (1 - p_i)^n (1 - p_j)^n) $