[Math] how to determine transient and recurrent state from transition matrix

markov chainsprobabilityprobability distributionsprobability theorystochastic-processes

I wonder how can I determine the transient and recurrent state from transition matrix ? I mean if I have 10 states It would be very hard to draw diagram for them so how to analyse the matrix?
For example consider :
$$
\begin{bmatrix}
0.1 & 0.2 & 0.7 & 0 \\
0.7 & 0 & 0.2 & 0.1 \\
0.6 & 0.4 & 0 & 0\\
0 & 0.5 & 0.5 & 0
\end{bmatrix}
$$
Can anyone tell me how to approach this problem I only want to know how to look at transition matrix and identify transient and recurrent states?

Thanks.

Best Answer

The formal way to do this and as defined in the book Introduction to Probability Models by Sheldon Ross is:

A state $i$ is recurrent if $\sum_{n=1}^{\infty}p_{ii}^n = \infty$

A state $i$ is transient if $\sum_{n=1}^{\infty}p_{ii}^n < \infty$

You can also define this as:

A state $i$ where $i \in S$ is said to be recurrent if $f_i$ = $P$(ever returns to $i$| starts in $i$) = 1

A state $j$ where $j \in S$ is said to be recurrent if $f_j$ = $P$(ever returns to $j$| starts in $j$) < 1

You would need to do this for all states in your matrix and solve it.