I want to compute the posterior distribution for a Jeffreys prior of a normal with unknown mean (and known variance $\sigma^2$)
My thoughts
Given a normal distribution $N(\mu,\sigma^2)$ with unknown $\sigma^2$, we have that the Fisher information is $I(\mu) = 1/\sigma^2$. Therefore, the Jeffreys prior should be $\pi(\mu) \propto 1/\sigma$.
It is clear that this is an improper distribution since $\int_{\mathbb{R}} 1/\sigma d\mu = \infty$ and as usually this is marked adding a positive constant $c$. I write $\pi(\mu) = c/\sigma$ with $c > 0$.
For the posterior distribution I compute $\pi(\mu|x) = \frac{f(x|\mu)\pi(\mu)}{\int_{\mathbb{R}} f(x|\mu)\pi(\mu) d\mu} = \frac{f(x|\mu)}{\int_{\mathbb{R}} f(x|\mu) d\mu}= f(x|\mu)$ since the denominator integrates one as it is the density function of a probability distribution.
However, I notice that this is not a normal distribution since it represents a conjoint distribution of a sample (here x is a vector). I read in a text that the posterior is a normal distribution with mean $\overline{x}$ and variance $\sigma^2/n$. Why is it the case?
Partial solution
I have encountered this situation previously and I think the trick will work here:
$f(x|\mu)\pi(\mu) = B(\sigma,n,c) \frac{1}{\sqrt{2\pi}\sigma^2/n} e^\frac{-(\overline{x}-\mu)}{2 \sigma^2/n}$
if i integrate with respect to $\mu$ then I get that the marginal is equal to the function $B(\sigma,n,c)$ which since $\sigma^2$ is known can be considered as a constant function.
Do you think this is the right answer?
Best Answer
According to page 3 here the Jeffrey's prior is $P[\mu] = c\sqrt{\frac{n}{\sigma^2}}$, which is still a constant so it doesn't matter.
Noting that $P[\bar{X}|\mu] = \frac{1}{\sqrt{2\pi\frac{\sigma^2}{n}}}\textrm{Exp}[\frac{(\bar{X}-\mu)^2}{2\frac{\sigma^2}{n}}]$.
Multiplying these two expressions together gives us:
\begin{equation} P[\bar{X},\mu] = \frac{c\sqrt{\frac{n}{\sigma^2}}}{\sqrt{2\pi\frac{\sigma^2}{n}}}\textrm{Exp}[\frac{(\bar{X}-\mu)^2}{2\frac{\sigma^2}{n}}] \end{equation}
Note that all of the stuff outside of the exponential is just a constant, so examining only the exponential kernel, we can see that the expression is a normal kernel for $\mu | \bar{X} \sim \textrm{N}(\bar{X},\frac{\sigma^2}{n})$.
So this is our posterior distribution.