Solved – Deriving the posterior density for a lognormal likelihood and Jeffreys’s prior

bayesianconjugate-priorposteriorprioruninformative-prior

The likelihood function of a lognormal distribution is:

$f(x; \mu, \sigma) \propto \prod_{i_1}^n \frac{1}{\sigma x_i} \exp \left ( – \frac{(\ln{x_i} – \mu)^2}{2 \sigma^2} \right ) $

and Jeffreys's Prior is:

$p(\mu,\sigma) \propto \frac{1}{\sigma^2} $

so combining the two gives:

$f(\mu,\sigma^2|x)= \prod_{i_1}^n \frac{1}{\sigma x_i} \exp \left ( – \frac{(\ln{x_i} – \mu)^2}{2 \sigma^2} \right ) \cdot \sigma^{-2} $

I know that the posterior density for $\sigma^2$ is inverse Gamma distributed, so I have to calculate

$f(\sigma^2|x) = \int f(\mu,\sigma^2|x) d\mu $

but I have no clue where to start here.

After Glen_b's comment I give it a shot:

$f(\mu,\sigma^2|x)= \prod_{i_1}^n \frac{1}{\sigma x_i} \exp \left ( – \frac{(\ln{x_i} – \mu)^2}{2 \sigma^2} \right ) \cdot \sigma^{-2} $

$= \sigma^{-n-2} \prod_{i=1}^n \frac{1}{x_i} \exp \left ( – \frac{1}{2\sigma^2} \sum_{i=1}^n (\ln x_i – \mu ) \right) $

but I cannot see this going anywhere.

Another idea I got is to define $y_i=\ln(x_i)$, then $y$ is normal distributed.
So

$f(\mu,\sigma^2 |y) = \left [ \prod_{i=1}^n \frac{1}{\sqrt{2 \pi}} \cdot \frac{1}{\sigma} \exp \left ( – \frac{1}{2 \sigma^2} (y_i – \mu)^2 \right ) \right ] \cdot \frac{1}{\sigma^2}$

$
\propto \sigma^{-n-2} \cdot \exp \left ( – \frac{1}{2 \sigma^2} \sum_{i=1}^n (y_i – \bar y)^2 + n(\bar y – \mu)^2 \right )
$
$
= \sigma^{-n-2} \cdot \exp \left ( – \frac{1}{2 \sigma^2} ( (n-1)s^2 + n(\bar y – \mu)^2 ) \right )
$
$
= \sigma^{-n-2} \cdot \exp \left ( – \frac{1}{2 \sigma^2} ( (n-1)s^2 \right ) \exp \left (n(\bar y – \mu)^2 ) \right )
$

then integrate:

$
\sigma^{-n-2} \cdot \exp \left ( – \frac{1}{2 \sigma^2} ( (n-1)s^2 \right ) \int \exp \left ( – \frac{1}{2 \sigma^2} n(\bar y – \mu)^2 ) \right ) d \mu
$

by the method you suggested I get:

$
\int \exp \left ( – \frac{1}{2 \sigma^2} n(\bar y – \mu)^2 ) \right ) d \mu = \sqrt{\frac{2\pi \sigma^2}{n}}
$

So:

$
\propto (\sigma^2)^{-(n+1)/2} \exp \left ( – \frac{1}{2 \sigma^2} ( (n-1)s^2 \right )
$

which is indeed inverse Gamma distributed.

But I am unsure if this is correct, it's also the same result as I get for a normal likelihood.

I found this in the literature (without any further explanantion):

enter image description here

Best Answer

Note that - regarded as a function in $\mu$ - what you have is proportional to a normal density.

So step 1 is to complete the square in $\mu$ that's in the exponent, pull out the front of the integral any superfluous constants, and then multiply the term in the integral by the constant required to make it integrate to 1. Then divide out in front of the integral by the same constant (so you don't change the value of the overall expression).

Since you have a density in the integral, replace the term in the integral by 1.

You're left with a function of $\sigma$ (one that has notionally replaced $\mu$ with something akin to an estimate of it).

Now see the density for an inverse gamma here:

$$f(x; \alpha, \beta)= \frac{\beta^\alpha}{\Gamma(\alpha)}x^{-\alpha - 1}\exp\left(-\frac{\beta}{x}\right)$$

(in this case, using a shape-scale parameterization).

Assuming you have the prior correct (I haven't checked that) --

you seek a posterior density for $\sigma^2$. Note that your function from after the integration can be written in the form $c\cdot(\sigma^2)^{-\text{something}}\cdot\exp(-\text{something-else}/\sigma^2)$.

So you have an expression proportional to an inverse gamma density in $\sigma^2$. (Since it must be a density, supply the required constant needed to make it integrate to 1.)