The standard or canonical form of the transition matrix of an absorbing Markov chain is
$$P = \begin{bmatrix}I & 0 \\ R & Q \end{bmatrix}$$
and the fundamental matrix is calculated as
$$F=(I – Q)^{-1}$$
such that $FR$ spells out the probability of eventually landing on each absorbing states from different transient states. At the same time, $F$ provides the expected number of steps required.
What is the proof that $F$ and $FR$ include this information?
Best Answer
The submatrix $R$ contains the transition probabilities from transient to absorbing states, while $Q$ contains the transition probabilities from transient to transient states.
Powers of the transition matrix $P$ approach a limiting matrix with a pattern:
$$\begin{align} P^2 &=\begin{bmatrix}I & 0 \\ R & Q \end{bmatrix}^2= \begin{bmatrix}I & 0 \\ R+QR & Q^2 \end{bmatrix}\\[3ex] P^3 &=\begin{bmatrix}I & 0 \\ R+QR & Q^2 \end{bmatrix}\begin{bmatrix}I & 0 \\ R & Q \end{bmatrix}=\begin{bmatrix}I & 0 \\ R+QR+Q^2R & Q^3 \end{bmatrix}\\[3ex] P^k &=\begin{bmatrix}I & 0 \\ \left(I+Q+Q^2+\cdots+Q^{k-1}\right)R & Q^k \end{bmatrix}\tag 1 \end{align}$$
The key now is that $Q^k\to 0$ as $k\to \infty.$
The fundamental matrix is a geometric series:
$$F= I+Q+Q^2+\cdots=(I-Q)^{-1}$$
and replacing the expression $I+Q+Q^2+\cdots+Q^{k-1}$ in (1) with $F:$
$$\begin{align}P^{\infty}&=\begin{bmatrix}I & 0 \\ FR & 0 \end{bmatrix}\end{align}$$