Solved – Time Reversible Markov Chain and Ergodic Markov Chain

markov-process

I am recently studying Markov Chain. However, I encountered one question regarding the difference between Ergodic Markov chain and time reversible Markov chain. My question is:

What is the difference between time reversible Markov chain and Ergodic Markov chain?

Best Answer

If I take the definition of David Aldous and Jim Fill, a finite state space Markov chain is time-reversible if it satisfies the detailed balance equation $$\pi_i\,p_{ij}=\pi_j\,p_{ji}$$ where the $p_{ij}$'s are the terms of the Markov transition matrix and the $\pi_i$'s are the terms of a probability distribution. Then, by summing both sides of the equation in $i$, we derive that $$\sum_{i=1}^N \pi_i\,p_{ij}=\sum_{i=1}^N \pi_j\,p_{ji}=\pi_j\,\sum_{i=1}^N p_{ji}=\pi_j$$ which implies that $(\pi_1,\ldots,\pi_N)$ is a stationary distribution for the Markov transition. If the chain is assumed to be irreducible, then the stationary distribution is unique. And the Markov chain is then ergodic if it is aperiodic.

The converse is not true, in that there exist non-reversible ergodic Markov chains. An example is provided by the Gibbs sampler associated with a vector $(X_,X_2,X_3)$ and a stationary distribution $P(x_1,x_2,x_3)$. Considering the transition from $\mathbf{X}^t$ to $\mathbf{X}^{t+1}$

  1. Generate $X_1^{t+1}\sim P(x_1|X_2^t,X_3^t)$
  2. Generate $X_2^{t+1}\sim P(x_1|X_1^{t+1},X_3^t)$
  3. Generate $X_3^{t+1}\sim P(x_1|X_1^{t+1},X_2^{t+1})$

is not time-reversible, but if the three conditional distributions have no restriction on their support, the resulting Markov chain is ergodic with distribution $P$.

Related Question