Solved – What would be the reason that the posterior distribution looks like the prior using MCMC

Question

I am trying to use MCMC to obtain the posterior probabilities of the free parameters of a model. I have tried first to leave two free parameters for my model and I was able to estimate the posterior probability for these free parameters using EMCEE. The reason that I am confident about the results is that I have tested them on simulated data and the peaks of posteriors were very close to true values.

Now, I have set four free parameters for this model but the posteriors that I have obtained is basically similar to my priors. I have chosen uniform priors for two new parameters that I left free. The posteriors for these two free parameters got uniform distributions with some spikes.

I have to mention that the characteristic property of the data is that it is very noisy. I am wondering whether I have messed up with the new likelihood that I set up for four parameters or whether MCMC can not constrain the free parameters given the property of the data (being noisy) or I need to use special type of resampling to obtain the proper posterior?

Update:
Here is a file contain the information of all points have been sampled by parallel tempering method. I have used 20 temperatures, 500 walkers and for 200 times iterations. I flatted the whole resampled data for different temperatures.

In the following I computed the likelihood for the gridded values of two new parameters fixing two other parameters. The maximum value of likelihood is in agreement with the true value in the mock data. However when I leave all four parameters free and use MCMC to get the posteriors for them it fails.

I have another question: The quantity that I measure with the model is the orientation of each object with respect to the center for the two given free parameters of the model. However, each object has an intrinsic orientation which is the main source of error in this measurement. At the moment, I use the rms value of all the object's orientations as $\sigma$ in the likelihood but how could I include the effect of each individual intrinsic orientation of object in the likelihood in order to get a better un-biased estimator?

Answer 1

Assuming your likelihood and MCMC work correctly, which we can't know of course, and assuming your new variables are not ineffective:

• If you increase the number of parameters under calibration with data fixed, an increase in marginal posterior uncertainty up to the point that the marginal posterior looks like the prior is quite common.

• This happens if your data is not any more informative enough to constrain all parameters at once to a particular value (i.e. you have trade-offs in parameter space = equifinality)

• A way to check this is to look at the correlation between parameters in the posterior - typically, you will see that while the marginal posteriors look flat or like the prior, the the multivariate posterior space is smaller than the multivariate prior space (i.e. you see correlations between the parameters that were not in the prior).

• A further check would be to compare the prior predictive distribution to the posterior predictive distribution.

• I'm sure there are better references, but an example of this phenomenon is in the appendix of 1, where we decrease the information in the data, and you see how marginal posteriors and correlations increase. For a moderate decrease of information in the data, you can still see the reduction of uncertainty in the posterior pair correlations, but if you further increase the information in your data, you get higher-order correlations and all looks random (it likely isn't though).

1 Hartig, F.; Dislich, C.; Wiegand, T. & Huth, A. (2014) Technical Note: Approximate Bayesian parameterization of a process-based tropical forest model. Biogeosciences, 11, 1261-1272. http://www.biogeosciences.net/11/1261/2014/bg-11-1261-2014.html

Addition: I thought for further reference, an illustration would be useful. What are are talking about is this type of correlation in posterior space:

library(psych)
par1= runif(1000,0,1)
par2 =par1 + rnorm(1000,sd = 0.05)
scatter.hist(par1,par2)

As one can see, the marginals look flat, but there is actually a large reduction in uncertainty in the two-dimensional space.