[Math] Find steps to reach absorbing markov chain state

linear algebramarkov chainsmatricesprobability

How can I find the steps it takes or days or whatever the time variable is till the matrix reaches the absorbing state.

e.g. take the matrix (The probability of each row adds to 1)
$$
\left[
\begin{array}{ccc}
0.3 & 0.5 & 0.2 \\
0.6 & 0.4 & 0 \\
0 & 0 & 1 \\
\end{array}
\right]
$$

please show the process

Best Answer

You can't find "the steps it takes" because that's a random quantity. But you can find the expected number of steps it takes, starting in a given state. In your example, let $u_1$ and $u_2$ be the expected number of steps until reaching the absorbing state, starting in states $1$ and $2$ respectively.

Suppose, for example, you start in state $1$.

The first column of the matrix says that after the first step, with probability $0.3$ you stay in state $1$ (and then the expected number of additional steps until reaching the absorbing state is again $u_1$); with probability $0.6$ you are in state $2$ (and then the expected number of additional steps is $u_2$), and with probability $0.1$ you are in the absorbing state $3$ so no additional steps are needed. This says $u_1 = 1 + 0.3 u_1 + 0.6 u_2$. Do a similar analysis for starting in state $2$, getting another equation, and solve the system of two equations for $u_1$ and $u_2$.

Related Question