Assume $X\sim \mathrm{iid}\operatorname{Pareto}(a,b)$, and $b \le \min(X)$, then
$$f_n({\bf x}; \theta) = a^n b^{-n} \prod^n_{i=1}\left( \frac{b}{x_i} \right) ^{1 + a} $$
We assume $b$ is known and choose a Gamma($\alpha, \beta$) prior distribution for $a.$
I aim to show the posterior distribution is a Gamma($\alpha + n, \beta + \sum^n_{i=1} \ln \frac{x_i}{b}$).
I begin with;
$$p(a\mid {\bf x}) \propto a^n b^{-n} \prod^n_{i=1}\left( \frac{b}{x_i} \right) \frac{\beta^\alpha}{\Gamma(\alpha)}a^{\alpha – 1}\exp(-\beta a)$$
$$\propto a^{(\alpha + n) – 1} b^{-n} \frac{\beta^\alpha}{\Gamma(\alpha)}\exp\left(-\beta a + \ln\prod^n_{i=1}\left( \frac{b}{x_i} \right) ^{1 + a} \right)$$
$$\propto a^{(\alpha + n) – 1} b^{-n} \frac{\beta^\alpha}{\Gamma(\alpha)}\exp\left(-\beta a + (1 + a)\sum^n_{i=1}\ln\left( \frac{b}{x_i} \right) \right)$$
$$\propto a^{(\alpha + n) – 1} b^{-n} \frac{\beta^\alpha}{\Gamma(\alpha)}
\exp \left(-a \left(\beta + \sum^n_{i=1} \ln \left(\frac{x_i}{b}\right)\right) + \sum^n_{i=1}\ln\left( \frac{b}{x_i}\right)\right) $$
$$\propto a^{(\alpha + n) – 1} \frac{\beta^\alpha}{\Gamma(\alpha)} \prod^n_{i=1}\left( \frac{1}{x_i} \right)
\exp \left(-a \left(\beta + \sum^n_{i=1} \ln \left(\frac{x_i}{b}\right)\right)\right) $$
Which is close. I have a to the correct power, and the term in the exponential is what I would like also. Since there is no a term in the other terms;
$$\propto a^{(\alpha + n) – 1}
\exp \left(-a \left(\beta + \sum^n_{i=1} \ln \left(\frac{x_i}{b}\right)\right)\right) $$
Which is what we want. But is it okay to disregard those other terms?
Best Answer
Small point: You need $\dfrac{\beta^{n\alpha}}{(\Gamma(\alpha))^n}$, where you have the $n$th root of that.
Now notice what "proportional to" means in this case: It means proportional as a function of $a$. Factors that don't depend on $a$ get absorbed into the normalizing contant, and should be disregarded because the way to find the normalizing constant is just by using what you know about Gamma densities. "Factors" means things you multiply be, as opposed to "terms", which I use to mean things you add or subtract. In Bayes' theorem you generally disregard "constant" factors, and in this case "constant" means not depending on $a$.
Bayes' theorem is sometimes stated like this: multiply the prior by the likelihood; then normalize, getting the posterior. Suppose the prior probability that you're throwing a fair coin is $2/3$ and the prior probability that it's a biased coin with probability $1/5$ of "heads" is $1/3$. Then you've got $$ \underbrace{\left(\frac13,\frac23\right)}_{\text{prior}}\cdot\underbrace{\left(\frac15,\frac12\right)}_{\text{likelihood}}\propto(1,2)\cdot(2,5)=(2,10)\propto(1,5)\propto\underbrace{\left(\frac16,\frac56\right)}_{\text{posterior}}. $$ That last pair gives the posterior probabilities given that you got "heads". Notice how each time you see "$\propto$", you're discarding a factor that is the same in both components, and then normalizing only at the last step. That's how it's done.