Detailed balance and Semi detailed balance in a Markov chain

markov chainsstochastic-processes

Consider the following transition matrix

$$
T=
\left[ {\begin{array}{cccc}
\frac{1}{3} & \frac{1}{4} & \frac{1}{5} & \frac{1}{6}\\
\frac{1}{3} & \frac{1}{4} & \frac{1}{5} & \frac{2}{6}\\
\frac{1}{3} & \frac{1}{4} & \frac{3}{5}& 0\\
0 & \frac{1}{4} & 0& \frac{3}{6}\\
\end{array} } \right]
$$

of a Markov chain process
$$P_{t+1}=TP_t$$

The matrix $T$ is not symmetric, therefore I conclude that there is no detailed balance, and there is no equilibrium (the process is not reversible).

However, $T_{34}=T_{43}$.

Does this mean that there is semi-detailed balance?

what does it say about the equilibrium of the system?

Best Answer

Symmetry of the transition matrix is not a necessary condition for reversibility; consider $$P = \pmatrix{0&1/2&1/2\\1/4&1/2&1/4\\1/4&1/4&1/2}. $$ Since $P$ is irreducible and aperiodic, it has a unique stationary distribution $\pi$ satisfying $\pi=\pi P$. Since $\sum_i \pi_i = 1$ we have the system of equations \begin{align} -\pi_1 + 1/4\pi_2 + 1/4\pi_3 &= 0\\ 1/2\pi_1 -1/2\pi_2 +1/4\pi_3 &= 0\\ \pi_1 + \pi_2 + \pi_3 &= 1, \end{align} which yields $\pi_1=1/5$, $\pi_2=2/5$, $\pi_3=2/5$. Let $P^\star_{ij}=(\pi_i/\pi_j)P_{ij},$ then $$ P^\star = \pmatrix{0&1/4&1/4\\1/2&1/2&1/4\\1/2&1/4&1/2} = P^T, $$ so the detailed balance equations are satisfied and $P$ is reversible.

In general, $P$ is reversible if and only if $\Pi * P$ is symmetric, where $\Pi$ is the square matrix whose rows are identically $\pi$ and $*$ denotes component-wise multiplication (see this thesis for more details).

In your example, the stationary distribution for $S:=T^T$ is $$\pi = \pmatrix{33/137&36/137&50/137&18/137}. $$ (The transpose of $T$ is a row-stochastic matrix.)
We compute $$ \Pi^T * S = \left( \begin{array}{cccc} \frac{33}{137} & \frac{33}{137} & \frac{33}{137} & \frac{33}{137} \\ \frac{36}{137} & \frac{36}{137} & \frac{36}{137} & \frac{36}{137} \\ \frac{50}{137} & \frac{50}{137} & \frac{50}{137} & \frac{50}{137} \\ \frac{18}{137} & \frac{18}{137} & \frac{18}{137} & \frac{18}{137} \\ \end{array} \right) * \left( \begin{array}{cccc} \frac{1}{3} & \frac{1}{3} & \frac{1}{3} & 0 \\ \frac{1}{4} & \frac{1}{4} & \frac{1}{4} & \frac{1}{4} \\ \frac{1}{5} & \frac{1}{5} & \frac{3}{5} & 0 \\ \frac{1}{6} & \frac{1}{3} & 0 & \frac{1}{2} \\ \end{array} \right) = \left( \begin{array}{cccc} \frac{11}{137} & \frac{11}{137} & \frac{11}{137} & 0 \\ \frac{9}{137} & \frac{9}{137} & \frac{9}{137} & \frac{9}{137} \\ \frac{10}{137} & \frac{10}{137} & \frac{30}{137} & 0 \\ \frac{3}{137} & \frac{6}{137} & 0 & \frac{9}{137} \\ \end{array} \right), $$ which is not symmetric, and hence the Markov chain is not reversible.

For Markov chains, the semi-detailed balance condition is precisely the global balance condition $\pi = \pi P$.