I was doing a question set by my tutor asking to find an irreducible markov chain where all the states were transient.
It's my understanding that is a markov chain is irreducible, all the states must communicate with each other inside the class $C$, i.e
$$
(\forall i,j\in C)( \exists n_1 , n_2 \in\mathbb{N})\left[P_{i,j}^{n_1} ,P_{j,i}^{n_2}>0\right]
$$
Hence if the class is irreducible, every element in the state space should be able to communicate with each other.
But if a state $k$ is transient, then it won't return back to $k$?
Furthermore, the example she gave was:
- State space $\mathbb{N}$
- Transition probabilities : $P_{i,i+1}=1; \ \ \ \ \forall i \in \mathbb{N}$
I don't think this is even irreducible! Surely this is split up into classes $\{1\}\cup\{2\}\cup\{3\}\cup\ \cdots \cup\{n\}\cup \cdots $ ?
Have I just got my definitions wrong?
Best Answer
You are right. The MC that you have mentioned with $P_{i,i+1}=1$ is not irreducible. For example, you can not go from state $5$ to state $4$. It is transient though.
You can consider the follow MC. state-space $S:=\{0,1,2,\ldots\}$, $P_{0,1} =1$, $P_{i,i+1}=0.9$, and $P_{i,i-1}=0.1$ for $i=1,2,\ldots$. This is both irreducible and transient.