Markov Chain – How to Understand the Connection Between Markov Chain and Markov Chain Monte Carlo?

Question

I am trying to understand Markov chains using SAS. I understand that a Markov process is one where the future state depends only on the current state and not on the past state and there is a transition matrix that captures the transition probability from one state to another.

But then I came across this term :Markov Chain Monte Carlo. What I want to know is if Markov Chain Monte Carlo is in anyway related to Markov process that I describe above?

Answer 1

Well, yes, there is a relationship between the two terms because the draws from MCMC form a Markov chain. From Gelman, Bayesian Data Analysis (3rd ed), p. 265:

Markov chain simulation (also called Markov chain Monte Carlo or MCMC) is a general method based on drawing values of $\theta$ from appropriate distributions and then correcting those draws to better approximate the target posterior distribution, $p(\theta|y)$. The sampling is done sequentially, with the distribution of the sampled draws depending on the last value drawn; hence, the draws form a Markov chain.