I have a sample $X_1,…, X_n$ that are iid with $Pois(\theta)$ as their distribution where $Θ = ℝ^+$. Now, suppose that the prior distribution of $𝜃$ is $Exp(\lambda)$. I am given the solution as follows:
The likelihood function is:
$L(\theta) = \prod_{i=1}^{n} \frac{e^{-\theta}\theta^{x_i}}{x_i!} = \frac{e^{-n\theta} \theta^{\sum_{i=1}^{n}}}{\prod_{i=1}^{n} x_i!}$
As such, the posterior distribution is:
$f(\theta|\mathbf{x}) = \frac{\frac{e^{-n\theta} \theta^{\sum_{i=1}^{n}}}{\prod_{i=1}^{n} x_i!} \times \frac{1}{\lambda}e^\frac{-\theta}{\lambda}}{\int_0^\infty \frac{e^{-n\theta} \theta^{\sum_{i=1}^{n}}}{\prod_{i=1}^{n} x_i!} \times \frac{1}{\lambda}e^\frac{-\theta}{\lambda} d\theta}$
My question is, why is the prior distribution written as $\frac{1}{\lambda}e^\frac{-\theta}{\lambda}$ and not $\frac{1}{\lambda}e^\frac{-x}{\lambda}$? Isn't the probability density function of $Exp(\lambda)$ distribution $\frac{1}{\lambda}e^\frac{-x}{\lambda}$? So where did $\theta$ come from?
Best Answer
You are dealing with Bayesian Statistic. Your parameter (unknown parameter $\theta$) has its own distribution. In particular your prior is an $Exp(\lambda)$ and the variable is exactly $\theta$
Moreover, the solution can be derive in a faster way
$$\pi(\theta|\mathbf{x})\propto \pi(\theta)p(\mathbf{x}|\theta)$$
Thus
$$\pi(\theta|\mathbf{x})\propto e^{-n \theta}\theta^{\Sigma x} e^{-\frac{\theta}{\lambda}}=\theta^{\Sigma x}e^{-\theta(n+\frac{1}{\lambda})}$$
We immediately recognize that
$$ \bbox[5px,border:2px solid black] { \pi(\theta|\mathbf{x})\sim \mathrm{Gamma}\Big(\Sigma x+1;n+\frac{1}{\lambda}\Big) \qquad (1) } $$
All the calculations you showed are not necessary and tedious
In fact all the quantities not depending on $\theta$ can be removed because they are included in the normalization constant...also you denominator does not depend on $\theta$ because it is integrated over all $\Theta$
Having derived that the posterior (1) is a Gamma with the specified parameters, the normalization constant is immediately given, thus the complete expression of your posterior is
$$ \bbox[5px,border:2px solid black] { \pi(\theta|\mathbf{x})=\frac{(n+\frac{1}{\lambda})^{\Sigma x+1}}{\Gamma(\Sigma x+1)}\theta^{\Sigma x}e^{-\theta(n+\frac{1}{\lambda})} \qquad (2) } $$
In the density (2) the variable is $\theta \in (0;+\infty)$; $\lambda$ is a parameter, while $n$ and $\Sigma x$ are given observed data.