My model contains five parameters. I want to make Bayesian estimation, but the Bayes estimates can not be obtained in closed form. So, I used Metropolis-Hastings to generate MCMC samples from conditional posterior density of each parameter. The trace and Auto-correlation plots were used to evaluate the generated sample. The trace plots for four parameters are random and the Auto-correlation plots are decreasing whereas for the fifth parameter(I will referred as alpha1), the trace plot is not random and the lags in Auto-correlation plot is not decreasing.
Discussion about alpha1
The acceptance rate is 0.99905 (It is too high) and the density plot is multimodal like the following graph
Trace plot
Auto-correlation plot
when I change the variance of the proposal density (absolute normal density) from 0.0005 to 0.5 the Acceptance rate becomes 0.1515 and the trace plot is horizontal line and the density function becomes uni-modal
I read MCMC for multimodal posterior
and this chapter from this book Evaluating Markov Chain Monte Carlo (MCMC) Algorithms. I am beginner in MCMC, I would be appreciated if one can help me to detect the problem. Is it multi modal or the correlation between parameters are high.
Many thanks in advance.
Edit
mixture 5 Normal density and trace plots
mixture 3 Normal trace plots
mixture 3 Normal density plot
Best Answer
Multimodality is often a cause of poor convergence or lack thereof for MCMC algorithms. Without any detail available about the posterior you aim at approximating, it is hard to tell whether or not you are facing such a case, because the multimodality on the first (density) figure is most likely an artefact reflecting the very slow move of the (sub) Markov chain in the $\alpha$ space.
What is more worrying is the trace plot you produce for $\alpha$ when the scale of the proposal density moves to $0.5$ as it seems the sequence does not change at all. The density you obtain is again an artefact I believe, namely a single Normal density produced by the kernel density estimator of R.
I would suggest using a sequence of scales in the Metropolis proposal, from the lowest 5e-5 to the largest 5e-1 (or higher), chosen at random at each iteration, meaning the use of a mixture proposal in the Metropolis acceptance ration. This means using the sum of the five Normal proposals in this ratio, rather than the Normal with the scale that has been chosen for that iteration.