Suppose that following conditions are true:
$$ X\mid\Theta=\theta\sim N(0,\theta) $$
$$ \Theta\sim InvGamma(2,1) $$
We need to find a posterior distribution: $$ f\left(\Theta\mid X\right) $$ given the data $$ X=-0.45,-3.92,-5.85,2.83,0.3 $$
And find:
$$ E\left\lbrack\Theta\mid X\right\rbrack,Var\left\lbrack\Theta\mid X\right\rbrack $$
First of all i'd like to find a function for the posterior distribution so i'll use this relationship:
$$ f\left(\Theta\mid X\right)\propto f\left(X\mid\Theta\right)\cdot f\left(\Theta\right)=\prod_{i=1}^n\left(\frac{1}{\left(2\pi\theta\right)^{\frac{1}{2}}}\cdot\exp\left(-\frac{x}{2\theta^{}}\right)\right)\cdot\frac{1}{\Gamma\left(2\right)}\theta^{-3}\cdot\exp\left(\frac{-1}{x}\right) $$
$$ =\frac{1}{\left(2\pi\theta\right)^{\frac{n}{2}}}\cdot\exp\left(-\frac{n\overline{x}}{2\theta^{}}\right)\cdot\frac{1}{1}\theta^{-3}\cdot\exp\left(\frac{1}{x}\right)\propto\frac{1}{\left(2\pi\theta\right)^{\frac{n}{2}}}\cdot\exp\left(-\frac{n\overline{x}}{2\theta^{}}\right)\cdot\theta^{-3} $$
However I am not sure what to do here. The problems that I have are:
-
How do I find a distribution that matches this function? My reasoning for this is that if I do find one, I can find the expected value and variance for it quite easily.
-
I am not sure what to do with the mean X even if I do find one.
Thank you in advance.
Best Answer
I think your expression is slightly off, the gaussian likelihood should instead be:
$$ \prod_{i=1}^n \frac{1}{\sqrt{2 \pi \theta}} \exp \left (-\frac{X_i^2}{2\theta} \right ) \stackrel{\theta}{\propto} \theta^{-n/2} \exp \left (-\frac{1}{2 \theta}\sum_{i=1}^n X_i^2 \right ) = \theta^{-n/2} \exp \left (-\frac{n \overline{X^2}}{2 \theta}\right ) $$
Therefore, the posterior is proportional to: $$ f(\theta|X) \stackrel{\theta}{\propto} \theta^{-n/2} \exp \left (-\frac{n \overline{X^2}}{2 \theta}\right ) \theta^{-3} =\theta^{-(3+n/2)} \exp \left (-\frac{n \overline{X^2}}{2 \theta}\right ) =\theta^{-{(2+n/2)}-1} \exp \left (-\frac{(n \overline{X^2}/2)}{\theta}\right ) $$
The last equality is written in this form to hint at the following important observation: by inspection, this looks much like the density of an inverse-Gamma distributed random variable. Recall that for $Y \sim \text{Inverse-Gamma}(a,b)$, $$ f_Y(y) = \frac{b^a}{\Gamma(a)} y^{-a-1} \exp(-b/y) $$ and so we can take $a = 2+n/2$ and $b = \frac{n }{2}\overline{X^2}$, i.e. we have shown that $$ \theta|X \sim \text{Inverse-Gamma} \left ( 2+\frac{n}{2}, \frac{n }{2}\overline{X^2} \right ) $$ and in your specific case, $\frac{n }{2}\overline{X^2} = \frac{5}{2} \times 11.57806 = 28.94515$, so
$$ \theta|X \sim \text{Inverse-Gamma} \left ( 4.5, 28.94515 \right ). $$
From here you can use standard results about the inverse gamma to compute the posterior mean and posterior variance. From the wiki page, we have
$$ E[\theta|X] = \frac{b}{a-1} = \frac{28.94515} {3.5} = 8.270043 $$ and $$ \text{Var}(\theta|X) = \frac{b^2}{(a-1)^2(a-2)} $$