Let's consider the following Markov chain. If we start at state $2$, compute the probability to hit $4$ and the expected time until it happens.
The probability to hit $4$ in $n$ steps starting at $2$ is $Pr(X_n=4|X_0=2)=(\frac{1}{2})^n$ if $n$ is even, and $0$ if $n$ is odd. Then, the total probability of hitting 4 if we start at 2 will be$\sum_{n\geq1}Pr(X_n=4|X_0=2)=\sum_{i\geq1}(\frac{1}{2})^{2i}=\sum_{i\geq1}\frac{1}{4^i}=\frac{1}{3}$
How can I calculate $\mathbb{E}(T_{24})?$ I have tried considering the mean hitting times $m_{ij}$, so $m_{24}=1+\frac{1}{2}m_{14}+\frac{1}{2}m_{34}$ and $m_{34}=1+\frac{1}{2}m_{44}+\frac{1}{2}m_{24}=1+\frac{1}{2}m_{24}$.
The solution in my book is $m_{24}=m_{34}=3$, which does not seem intuitive to me.
Best Answer
I get $m_{24}=4$ and $m_{34}=3$. We are computing the expected time to hit state $4$ starting in state $2$, assuming that it happens. There is a positive probability that we never reach state $4$ so if we don't condition the expectation on success, it would be $\infty.$
Therefore, we have to adjust the state diagram. The arrow from state $2$ to state $1$ goes away, and the arrow from state $2$ to state $3$ gets weight $1$. Now we have $$\begin{align} m_{24}&=1+m_{34}\\ m_{34}&=1+\frac12 m_{24} \end{align}$$ which leads to the solutions I gave above. You may find the accepted answer to Expected time till absorption in specific state of a Markov chain instructive.