Per the Wikipedia on conjugate priors link, the conjugate prior for a Gamma of unknown $\alpha$ and $\beta$ is proportional to an expression involving both $\alpha$ and $\beta$ as well as a $\Gamma$ function and 4 parameters.
Previously, I have had experience using conjugate priors for multi-parameter distributions (namely Gaussian) by employing the technique described in this paper. With this method, I am able to set 2 different conjugate priors, one over $\mu$ and one over $\sigma^2$. The end result is that each parameter can be sampled from it's own posterior so I can make inferences about each separately.
Given the irregular form for the Gamma conjugate prior, and the fact that is described as proportional (so I suppose this isn't a proper posterior distribution?), is it possible to compare values of both $\alpha$ and $\beta$ across two different posteriors (using same prior but different data) in the same way I have done for the Gaussian, or even any simple one parameter distribution with conjugate priors that you can find in the same Wikipedia article? How would I actually sample from this posterior?
Best Answer
If you take a Gamma Ga$(\alpha,\beta)$ likelihood $$\dfrac{\alpha^{n\beta}}{\Gamma(\beta)^n}\exp\left\{\beta\sum_{i=1}^n\log x_i -\alpha \sum_{i=1}^n x_i\right\}$$ the distribution with density $$\pi(\alpha,\beta) \propto \dfrac{\alpha^{\lambda\beta}}{\Gamma(\beta)^\lambda}\exp\left\{\beta\xi -\alpha \mu\right\}$$ is conjugate, if non-standard.
Furthermore, the conditional of $\alpha$ given $\beta$ is a Gamma, hence can be integrated out. A posteriori and a priori. The symmetric property for $\beta$ given $\alpha$ does not hold though.