I've got the following Discrete-Time Markov Chain (DTMC). This is the transition matrix
$$\begin{bmatrix}1/4 & 0 & 3/4 & 0 \\ 0 & 1/3 & 0 & 2/3 \\ 0 & 1 & 0 & 0 \\ 0 & 2/5 & 0 & 3/5 \end{bmatrix}$$
Drawing out the graphical representation for this, I see that we can go from 0 to 2, then go from 2 to 1 (with probability 1). So state 2 is sort of an intermediate state between two communication classes.
So my question is, are the communication classes just {0} and {1,3}? Is {2} a communication class as well? I guess what I'm asking is, does every state have to belong to a communication class? Because, it doesn't seem like {2} can be a communication class. It doesn't communicate with anything.
Best Answer
State $2$ makes up one communicating class by itself. The relation $i \leftrightarrow j$ ($i$ communicates with $j$) is an equivalence relation; hence it defines a partition of the states. The number of steps required to go from $i$ to $j$ does not have to be strictly positive. Hence every state is in communication with itself. The class $\{2\}$ is open and transient.