In Metropolis-Hastings sampling it is required to have a proposal distribution $g(\theta_{new}|\theta_{old},\lambda)$, where $\theta_{old}$ is the last accepted sample in the chain and $\theta_{new}$ is a newly proposed sample. Samples are drawn from $g$ which are then transformed into the acceptance probability (the proposal step) for subsequent use in the move step. For example, $g$ is often chosen multivariate normal with mean $\theta_{old}$ and variance-covariance matrix $\lambda$.
I have a parameter $\theta$ which is a precision (or scale or variance) so that $\theta >0$ is a constraint on the parameter space. This thread discusses various options in this situation. I find the best option to use a function $g$ which only proposes samples from the allowed parameter space.
My questions are: which are commonly used distributions in this situation? And, how should the proposal distribution be conditioned on $\theta_{old}$?
For example, I can imagine the uniform $U(0,b)$ distribution may be useful, but I do not see how to choose $b$ or how to condition on $\theta_{old}$ (i.e. the last accepted precision/variance in my case).
An alternative may be Gamma or inverse Gamma, e.g. $Gam(a,b)$, but then also the question is how to choose $a$ and $b$ and how to link them to $\theta_{old}$.
Best Answer
The most natural [and generic] resolution [imo] is to turn $\theta$ into $\eta=\log\theta$ in the original problem so that $\eta$ is unconstrained. This allows for the use of random walk proposals like Metropolis et al.'s. The only warning is that the prior must incorporate the change of variable through a Jacobian: $$\pi_\eta(\eta)=\pi_\theta(\exp\{\eta\})\times\exp\{\eta\}$$ Warning: This proposal is only equivalent to propose a log-normal new value in the original parameterisation if the proper Metropolis ratio is used [in the original parameterisation, the proposal is no longer a random walk]. (Note that an exponential change of variables turns the likelihood $p(D|θ,ν)$ into $p(D|\exp\{η\},ν)$, without a Jacobian there!)
Otherwise, a Uniform $\text{U}(\theta^\text{old}-\epsilon,\theta^\text{old}+\epsilon)$ proposal, as in Hastings (1970), can be chosen as a proposal, with the potential to propose negative [and hence surely rejected] values.