Solved – How to incorporate parameter constraints in Metropolis Hastings

Question

I am working on parameter estimation of GARCH model with Metropolis Hastings. But the results I have got doesn't look reasonable, actually it is quite different from what I have got from Gibbs sampler. Since there is some upper bound constraints on some of the parameters, I use truncated Normal distribution(so that sample won't be drawn from outside the constraints) as the proposal distribution. Will this be the reason that the MH is not working well? But if I use a Normal with constraint, quite a number of samples will be discarded right away, plus it is not converging after 10k draws. I understand that the proposal distribution will impact the acceptance rate, but don't understand how distort the estimation. Any comment will be helpful.

Answer 1

If you use a truncated normal as your proposal, $\mathcal{N}_a^b(\mu,\sigma^2)$ where $(a,b)$ is the truncation interval,

1. If $a$ or $b$ depend on the previous value of the Markov chain, the normalisation constant $$\Phi(\{b-\mu\}/\sigma)-\Phi(\{a-\mu\}/\sigma)$$ must also appear in the acceptance probability $$\frac{\pi(x')}{\pi(x)}\times\frac{\varphi(x|a',b',\mu',\sigma')}{\varphi(x'|a,b,\mu,\sigma)}\wedge 1$$where $\varphi(x|a',b',\mu',\sigma')$ denotes the truncated normal density, i.e. $$\varphi(x|a,b,\mu,\sigma)=\dfrac{\exp\{-(x-\mu)^2/2\sigma^2\}\mathbb{I}_{[a,b]}(x)}{\sqrt{2\pi‌​}\sigma[\Phi(\{b-\mu\}/\sigma)-\Phi(\{a-\mu\}/\sigma)]}$$
2. The simulation from the truncated normal does not require discarding any value as you can use the inverted cdf technique, as explained in this earlier answer.