Question: Let $\{X_n:n \ge0 \}$ be a time- homogeneous discrete time Markov-chain with either finite or countable state space $S$. Then
$1.$ there is at least one recurrent state
$2.$ if there is an absorbing state, then there exists at least one stationary distribution
$3.$ if all states are positive recurrent, then there exists a unique stationary distribution
$4.$ If $\{X_n:n \ge0 \}$ is irreducible, $S=\{1,2\}$ and $[\pi_1 , \pi_2]$ is a stationary distribution, then $\lim_{n\to \infty} P\{X_n=i|X_0=i\}=\pi_i,$ for $i=1,2.$
My Attempt:
$1.$ I gave a counterexample. Take, $P=\left [ \begin{matrix} 1/2 & 1/2 &0 &0& \cdots \\ 0 & 1/2 & 1/2&0 & \cdots \\ \vdots& \vdots & \vdots& \vdots& \ddots \\ \end{matrix} \right ]$ with infinite state space $S$. Here, all states are transient.
$2.$ If there is an absorbing state, say state-$i$, so $d(i)=1$ and hence, state-$i$ is aperiodic. Since, every absorbing state is recurrent which implies Markov-chain has a recurrent, aperiodic state-$i$. I could not think further.
We first recall, "An irreducible positive recurrent Markov chain has a unique stationary distribution." See thisTheorem4,5.
$3.$ Take, $P= \left [ \begin{matrix} 1&0 \\ 1/2&1/2\\ \end{matrix} \right ]$. Here, all states are positive recurrent but it is infinitely stationary distributed.
$4.$ Really confused if we can apply theorem$5$.
The correct answer given in the key is $2$.
Kindly help me to prove this second option and help me understanding the last option( I am confused) and please check if my approach for other options is fine. Is there any use of the "homogeneous" thing here?
Thanks!
Best Answer
I have got the answers but I am answering to remove this question from unanswered que and for others if someone needs the answer in future.
For $(2)$, using Kavi Rama Murty's hint given in the comments:
If there is an absorbing satate-$i$ $\implies$ $i$ is a periodic, recurrent. We get, $\pi_j=0, \forall j\ne i$, and $\pi_i=1$. This gives a stationary distribution.
For $(4)$, use lonza leggiera's hint:
Given $\{X_n: n\ge0\}$ is irreducible with finite state space $S=\{1,2\}$ and it has a stationary distribution $\pi= [\pi_1, \pi_2]$. Note that the given chain may not be aperiodic, so we take a periodic chain with transition probability matrix $P=\left[ \begin{matrix} 0&1 \\ 1 &0\end{matrix} \right ]$. Here, $\left[ \begin{matrix} 1/2&1/2 \end{matrix} \right ]$ is a stationary distribution but this chain does not have a limiting distribution.