Say I have a Markov chain $\{X_n: n \geq 1\}$ with state space $E = \{1,2,3,4,5\}$ and transition matrix,
$$ P = \begin{bmatrix} 0 & 1/2 & 1/2 & 0 & 0 \\\ 1/2 & 0 & 1/2 & 0 & 0 \\\ 1/4 & 1/4 & 0 & 1/4 & 1/4 \\\ 0 & 0 & 1/2 & 0 & 1/2 \\\ 0 & 0 & 0 & 1/2 & 1/2\ \end{bmatrix} $$
With a stationary distribution,
$$ \pi^T = [1/6 \quad 1/6 \quad 1/3 \quad 1/6 \quad 1/6]$$
Assuming that $X_1 = 1$, it is easy to use this information to calculate the expected number of transitions between successive visits to state 1 (the answer is $1/\pi_1 = 6$}.
What is not clear to me, however, is how to calculate the expected number of transitions between successive visits to state 1 conditional on the fact that state 5 is never visited.
Two potential approaches to answer this are:
1) Constructing the Markov chain graph without state 5, recalculating the transition matrix and the stationary distribution. This approach yields a transition matrix of
$$ P = \begin{bmatrix} 0 & 1/2 & 1/2 & 0 \\\ 1/2 & 0 & 1/2 & 0 \\\ 1/3 & 1/3 & 0 & 1/3 \\\ 0 & 0 & 1 & 0\ \end{bmatrix} $$
a stationary distribution of
$$ \pi^T = [1/4 \quad 1/4 \quad 3/8 \quad 1/8]$$
and an answer of 4.
2) Calculating the answer as $(1/\pi_1)$ after conditioning the stationary distribution to account for the fact that state $5$ is not visited. In other words, we normalize $\pi_1 \ldots \pi_4$ by a factor of $1 – \pi_5$ to account for the fact that state $5$ is not visited. This yields a stationary distribution
$$ \pi^T = [1/5 \quad 1/5 \quad 2/5 \quad 1/5 \quad 0]$$
and an answer of $5$.
Some friends have argued that approach #1 is the correct approach, but cannot explain why approach #2 is wrong.
Any insight is appreciated.
Best Answer
The reason number 2 is wrong is that each number is predicated upon being able to either go to or come from 5 in each state, which does not uniformly affect each of the states. The degree to which each state's steady state varies depends upon the column corresponding to 5 in the transition matrix, which is not the same for each.