Solved – Use of the Jeffreys prior in multidimensional models

bayesianjeffreys-priorprior

Suppose a model,

$$x_{i} \sim N(\theta_{i}, \phi), \text{ for } i=1,\ldots,n$$

Furthermore, suppose the variance parameter, $\phi$, is some known constant.

The multidimensional Jeffreys prior is defined as,

$$\pi(\vec{\theta}) \propto \sqrt{\text{det}I(\vec{\theta})}$$

where $\text{det}I(\vec{\theta)}$ is the determinant of the multidimensional Fisher information.

Can a multidimensional Jeffreys prior be used to formulate a prior distribution for the posterior distribution, $p(\vec{\theta}|x_{1},\ldots,x_{n})$?

Best Answer

Can a multidimensional Jeffreys prior be used to formulate a prior distribution for the posterior distribution, p(θ⃗ |x1,…,xn)?

Yes, but it's usually not recommended to use Jeffreys prior for anything other than a single parameter model. The go-to approach for multidimensional cases is the Reference prior. Reference priors maximize the K-L divergence between the prior and the posterior averaged over the sufficient statistic distribution, so that we learn as much as possible from the data. A strong introduction and justification for Reference priors can be found here. A more approachable introduction is available in this lecture accompanied by this lecture. If it's comforting at all, for single parameter models the Reference prior works out to be the Jeffreys prior in each case.