I understand that a Markov chain is reversible if $\pi_{i}P_{ij} = \pi_{j}P_{ji}$ and I was looking for some examples of non-reversible Markov chains. I noticed that in all the examples I saw, at least one element in the main diagonal of the transition matrix was 0. I wonder if this is a coincidence or if there is actually a rule for this. Specifically, if all the elements in the main diagonal of a transition matrix are non-zero, is it possible that the chain is not reversible? If it is possible, could you please provide an example transition matrix?
Also, what is the physical interpretation of a Markov chain that has a stationary distribution but is not reversible? Essentially this means that the chain obeys the balance condition but not the stricter detailed balance condition, but I can't figure out an intuitive example.
Best Answer
$$P =\left[\matrix{ 9/10 & 0 & 1/10\\9/10 & 1/10 & 0\\ 0 & 9/10 & 1/10}\right]\\ $$
This transition matrix is irreducible (so it has a stationary strictly positive distribution), but it is not reversible, simply because given the invariant distribution $\pi$, you have $\pi_1P_{1,2}=0$, while $\pi_2 P_{2,1}\neq 0$.
The intuition is that if you look at the evolution forward in time, the Markov chain will look like it will be stuck in 1 most of the time, but once in a while it will jump to 3, where it will likely jump down to 2 and then to 1 with high probability, while sometimes it might be stuck in 3 or 2 for a little bit more. If you look at the process backwards in time, you will see essentially that the role of 2 and 3 are inverted, that is, if you look at a video of the Markov chain, you can really tell whether the time is going forward or backwards.
As you can tell, it would be easier just to put zeroes in the diagonal at entries 2,2 and 3,3, but I made them non-zero to answer your question. I am sure you can modify the example in such a way that all the entries are nonzero, but then you would have to compute $\pi$ to conclude rigorously that it is not reversible.
Off topic. I suspect that the set of irreducible, non-reversible transition matrices on a finite set is open, so that if you perturb the coefficients of $P$ slightly, then you obtain another irreducible, non-reversible transition matrix. So if you pick a “random” Markov chain, I guess it will be non-reversible (if you have at least 3 states).