Consider a Markov chain with transition matrix
$ \begin{matrix}
\frac{1}{3} & \frac{1}{3} & \frac{1}{3} & 0 & 0 & 0 & 0 \\
\frac{2}{3} & 0 & \frac{1}{3} & 0 & 0 & 0 & 0 \\
0 & \frac{2}{3} & \frac{1}{3} & 0 & 0 & 0 & 0 \\
0 & \frac{1}{4} & 0 & \frac{1}{2} & 0 & \frac{1}{4} & 0 \\
0 & 0 & \frac{1}{4} & 0 & \frac{1}{4} & 0 & \frac{1}{2} \\
0 & 0 & 0 & 0 & 0 & 0 & 1 \\
0 & \frac{1}{8} & \frac{1}{8} & \frac{1}{8} & \frac{1}{8} & \frac{1}{4} & \frac{1}{4}
\end{matrix} $
Say the states space are $\{1,2,3,4,5,6,7 \}$.Identify the communication classes. Classify the states as recurrent or transient. For all $i$ and $j$, determine $\lim_{n \to \infty} P_{ij}^n$ without using technology.
I can identify the communication classes are $\{1,2,3\}$ and $\{4,5,6,7 \}$. But I don't know how to judge whether the states are recurrent or transient.
Best Answer
HINT
Given two states $i\in\mathcal{S}$ (where $\mathcal{S}$ denotes the state space) and $j\in\mathcal{S}$, we say that $j$ is accessible by $i$ (denoted by $i\to j$) if there is a path from $i$ to $j$ in the state transition diagram of the corresponding finite state Markov chain. Based on it, we say that $i\in\mathcal{S}$ is recurrent if, and only if, for every $j\in\mathcal{S}$ such that $i\to j$ it also holds that $j\to i$. Otherwise, we call it transient. Finally, being recurrent or transient is a property of the state class.
Can you gather the previous comments in order to solve the proposed exercise?