[Math] Posterior mean of Dirichlet distribution

probabilityprobability distributions

Given the multinomial distribution: $$P[X_1 = x_1,\ldots,X_k = x_k|n,\theta_1,\ldots,\theta_k] = \frac{n!}{x_1!\ldots x_k!} \prod_{i = 1}^k \theta_i^x$$ where $\sum_{i = 1}^k \theta_i = 1,\sum_{i = 1}^k x_i = n$ and a sample $x = (x_1,\ldots,x_k)$

I want to compute the posterior mean of $\theta_1$ using the conjugate family.

My reasoning

The multinomial distribution is $p(x|\theta) = \frac{n!}{\prod_i x_i!} \prod_i \theta_i^{x_i}$ and the Dirichlet distribution is $p(\theta|\alpha) = \frac{1}{B(\alpha)}\prod_i \theta_i^{\alpha_i-1}$. Now I should compute the posterior distribution which seems to be

$$p(x|\theta) p(\theta|\alpha) \propto \prod_i \theta_i^{x_i+\alpha_i-1}$$

So I can assume that the posterior $p(\theta|x,\alpha')$ is a Dirichlet distribution with parameters $x_i+\alpha_i$.

The posterior mean would be $\int_{\theta} \theta p(\theta|x,\alpha') d\theta$

My question

How does one finish this exercise, namely how does one integrate the last expression?

Best Answer

The posterior distribution is a distribution in $\theta$ with parameters $x_i + \alpha_i$. This amounts to compute the mean of a particular Dirichlet distribution. This is done here. Therefore: $E_{\pi(\theta|x)} \theta_j = \frac{x_j + \alpha_j}{\sum (x_i+\alpha_i)} = \frac{x_j + \alpha_j}{n + \sum \alpha_i}$.

Related Question