I have a question regarding the definition of the equivalence relation leading to the so called communication classes. Let's assume we are given the following transition matrix
$$ \begin{equation*}
P =
\begin{pmatrix}
0.5 & 0.5 & 0 & 0 & 0 & 0 \\
0.3 & 0.7 & 0 & 0 & 0 & 0 \\
0 & 0 & 0.1 & 0 & 0.9 & 0 \\
0 & 0 & 0.7 & 0 & 0.3 & 0 \\
0 & 0.2 & 0 & 0.2 & 0.2 & 0.4
\end{pmatrix}
\end{equation*} $$
I draw the graph and see that there are three communication classes $\{0,1\},\{2,4\}$ and $\{3,5\}$. I have a question regarding recurrent and transient states. A state $i$ is called transient if
$$f_i:= P(\inf\{t\ge 1:X_t=i\}<\infty|X_0=i)<1 $$
and recurrent if
$$f_i:= P(\inf\{t\ge 1:X_t=i\}<\infty|X_0=i)=1 $$
I'm a bit puzzled how these definitions work. Take the class $\{0,1\}$. The only way to not get back to $0$ is jumping to $1$ and then stay there for ever, i.e. probability $0.5 + 0.7^n > 0$. That would imply there are no recurrent states but all states are transient. Am I'm misunderstanding the definition in this case? What are the transient / recurrent states if they differ from my solution
Best Answer
Probability that you start at $0$ and don't return to $0$ by time $n$ is $(0.5)(0.7)^{n}$. As $ n \to \infty$ this tends to $0$. So the probability of never returning to $0$ is $0$ which means you will return to $0$ with probability $1$.