I am working in a Bayesian framework: I have some observations $y$, for which I assume a statistical model. The model depends on parameters $\theta \in \Theta$ ($\Theta$ is the parameters space). I assume a probability distribution $q$ on $\Theta$. The parameters of this model can be estimated in a maximum a posteriori fashion :
$$ \hat{\theta} = \mathop{\mathrm{argmax}} \limits_{\theta \in \Theta} p(\theta \mid y) $$
where $p(\theta \mid y)$ is the posterior distribution of $\theta$ given $y$.
Now, say I want to perform a change of variable on $\Theta$. I consider a mapping $g \, : \, \Theta \, \rightarrow \, \Theta$ which transforms the "old" parameters $\theta$ in $\theta^{\mathrm{new}} = g(\theta)$. We assume that $g$ is a smooth diffeomorphism. My question is : how does this change of variable modifies the posterior distribution $p(\theta \mid y)$ ?
If $\mathrm{J}_{g}(\theta)$ denotes the jacobian matrix of $g$ at $\theta$, we know that $\theta^{\mathrm{new}}$ has a probability distribution $\widetilde{q}$ on $\Theta$ given by :
$$ \widetilde{q}(\theta^{\mathrm{new}}) \vert \mathrm{J}_{g}(\theta) \vert = q(\theta). $$
Using Bayes formula, I would write :
$$ p(\theta^{\mathrm{new}} \mid y) = \frac{ p\big( y \mid \theta^{\mathrm{new}} \big) \widetilde{q}(\theta^{\mathrm{new}}) }{ p(y) } = \frac{ p\big( y \mid g(\theta) \big) \vert \mathrm{J}_{g}(\theta) \vert^{-1} q(\theta) }{ p(y) }. $$
Is this correct or am I mistaken?
Best Answer
The change of variable in the posterior density is a standard change of variable, involving the Jacobian. The impact on the maximum a posteriori estimator is thus significant in that the MAP of the transform is not the transform of the MAP. (There are deeper reasons for disliking MAP estimators, of course.)