Let $(X_1, \dots, X_k) \sim Dir(\alpha_1, \dots, \alpha_k)$. If we let $Z_1 = X_1+X_2$, what would the joint distribution of $X_1, Z_1$ be? Same with the posterior distribution of $X_1$ given $Z_1$?
I know that the distribution of $Z_1$ is $Dir(\alpha_1+\alpha_2, \sum_{i=3}^{k} \alpha_i)$ and that the distribution of $X_1$ is $Beta(\alpha_1, \sum_{i=2}^{k} \alpha_i)$, but I don't know how to combine the two to get their joint distribution.
Based on the properties I've found, I'd have guessed something like $(X_1, Z_1, 1-X_1-Z_1) \sim Dir(\alpha_1, \alpha_1+\alpha_2, 1-2\alpha_1-\alpha_2$), but the fact that the $\alpha_1$ is repeating doesn't seem right to me
Best Answer
The distribution of $(X_1, X_2)$ is given by $\text{Dir}(\alpha_1, \alpha_2, \sum_{i = 3}^{n} \alpha_i)$ (proof below). Therefore the joint density of $(X_1, X_2)$ is $$f_{X_1, X_2}(x_1, x_2) = \frac{\Gamma(\sum_{i = 1}^{n}\alpha_i)}{\Gamma(\alpha_{1})\Gamma(\alpha_{2})\Gamma(\sum_{i = 3}^{n}\alpha_i)}x_1^{\alpha_1 - 1}x_2^{\alpha_2 - 1}(1 - x_1 - x_2)^{(\sum_{i = 3}^n \alpha_i) - 1},$$ where $x_1, x_2 > 0$ and $x_1 + x_2 < 1$.
Now we do a transformation of variables: Let $Z = X_1 + X_2$ and $X = X_1$ (I'm using $Z, X$ instead of $Z_1, X_1$ because, later on, the subscripts look messy otherwise). Then $$f_{X, Z}(x, z) = f_{X_1, X_2}(x_1(x, z), x_2(x, z))\left|\det\left(\frac{d(x_1, x_2)}{d(x, z)} \right) \right|.$$ The Jacobian here is $1$ and so the joint density is $$f_{X, Z}(x, z) = \frac{\Gamma(\sum_{i = 1}^{n}\alpha_i)}{\Gamma(\alpha_{1})\Gamma(\alpha_{2})\Gamma(\sum_{i = 3}^{n}\alpha_i)}x^{\alpha_1 - 1}(z - x)^{\alpha_2 - 1}(1 - z)^{(\sum_{i = 3}^n \alpha_i) - 1},$$ where $x > 0$, $z > x$ and $z < 1$.
The marginal density of $Z$ is $\text{Dir}(\alpha_1 + \alpha_2, \sum_{i = 3}^{n} \alpha_i)$, so the conditional density of $X$ given $Z$ is $$f_{X \mid Z}(x \mid z) = \frac{f_{X, Z}(x, z)}{f_{Z}(z)} = \frac{\frac{\Gamma(\sum_{i = 1}^{n}\alpha_i)}{\Gamma(\alpha_{1})\Gamma(\alpha_{2})\Gamma(\sum_{i = 3}^{n}\alpha_i)}x^{\alpha_1 - 1}(z - x_1)^{\alpha_2 - 1}(1 - z)^{(\sum_{i = 3}^n \alpha_i) - 1}}{\frac{\Gamma(\sum_{i = 1}^{n}\alpha_i)}{\Gamma(\alpha_{1} + \alpha_{2})\Gamma(\sum_{i = 3}^{n}\alpha_i)}z^{\alpha_1 + \alpha_2 - 1}(1 - z)^{(\sum_{i = 3}^n \alpha_i) - 1}},$$ which simplifies to $$f_{X \mid Z}(x \mid z) = \frac{1}{z}\frac{\Gamma(\alpha_1 + \alpha_2)}{\Gamma(\alpha_1)\Gamma(\alpha_2)}\left(\frac{x}{z}\right)^{\alpha_1 - 1}\left(1 - \frac{x}{z}\right)^{\alpha_2 - 1}.$$
Proof that the distribution of $(X_1, X_2)$ is given by $\text{Dir}(\alpha_1, \alpha_2, \sum_{i = 3}^{n} \alpha_i)$:
Let $I(x, k, n) = \{(x_k, \dots, x_n): x_k, \dots, x_n > 0, \sum_{i = k}^n x_i = x\}$. We know that $X_1 \sim \text{Beta}(\alpha_1, \sum_{i = 2}^{n} \alpha_i)$. Therefore $$\int_{I(1 - x_1, 2, n)} f_{X_1, \dots, X_n}(x_1, \dots, x_n) dx_2\cdots dx_n = \frac{\Gamma(\sum_{i = 1}^{n}\alpha_i)}{\Gamma(\alpha_{1})\Gamma(\sum_{i = 2}^{n}\alpha_i)}x_1^{\alpha_1 - 1}(1 - x_1)^{(\sum_{i = 2}^n \alpha_i) - 1}.$$ Hence $$\int_{I(1 - x_1, 2, n)} x_2^{\alpha_2 - 1}\cdots x_n^{\alpha_n - 1} dx_2\cdots dx_n = \frac{\Gamma(\alpha_2)\cdots\Gamma(\alpha_n)}{\Gamma(\sum_{i = 2}^n \alpha_i)}(1 - x_1)^{(\sum_{i = 2}^n \alpha_i) - 1}.$$ Therefore $$f_{X_1, X_2}(x_1, x_2) = \frac{\Gamma(\sum_{i = 1}^n \alpha_i)}{\Gamma(\alpha_1)\cdots\Gamma(\alpha_n)}x_1^{\alpha_1 - 1}x_2^{\alpha_2 - 1} \int_{I(1 - x_1 - x_2, 3, n)} x_3^{\alpha_3 - 1}\cdots x_n^{\alpha_n - 1} dx_2\cdots dx_n$$ $$= \frac{\Gamma(\sum_{i = 1}^{n}\alpha_i)}{\Gamma(\alpha_{1})\Gamma(\alpha_2)\Gamma(\sum_{i = 3}^{n}\alpha_i)}x_1^{\alpha_1 - 1}x_2^{\alpha_2 - 1}(1 - x_1 - x_2)^{(\sum_{i = 3}^n \alpha_i) - 1}$$