In a Markov chain with finite state space and absorbing states, if an absorbing state is reachable from every state, then it will absorb with probability $1$. The same does not hold true if there is an infinite state space (e.g. birth-death processes). I am looking to find this probability.
Let $M$ be the transition matrix so that each column adds up to $1$, the first $n$ states be the absorbing states ($p_{i \to i} = 1$), and the initial state be state $s$. Since $\sum_{m=1}^{n} (M^k)_{s \to m}$ is the probability of going from state $s$ to an absorbing state within $k$ steps, then $$\lim_{k \to \infty} \sum_{m=1}^{n} (M^k)_{s \to m}$$ should be the probability of absorption. Assuming this is correct, I run into a problem – how do I find $\lim_{k \to \infty} M^k$? This is raising an infinite matrix to $\infty$, which seems impossible.
My question:
What is the probability of eventual absorption in a Markov chain with infinite state space?
Best Answer
In principle this is a nontrivial computational problem. A way to handle it that works in specific, sufficiently simple problems is to write your problem as a limit of finite problems. You do that in the following way. Introduce an increasing sequence of finite subsets $X_n$ of the state space $X$ such that $\bigcup_{n=1}^\infty X_n=X$. Now introduce a new absorbing state, let's just call it $\infty$ just because it's a fun name, and take the probabilities to transition into $\infty$ from inside $X_n$ to be the sum of all the probabilities to exit $X_n$ from each state in $X_n$ in the original chain.
Now we pose a slightly different question: what is the probability to absorb to one of the absorbing states in $X_n$ instead of $\infty$? Obviously this depends on where you start, as well as on $n$, so we can call it $q_n(x)$, and the dependence on the starting point satisfies the equation
$$q_n(x) = 1 \quad x \in A \cap X_n \\ q_n(\infty)= 0 \\ q_n(x) = \sum_{m \in X_n} p_{xm} q_n(m) \quad x \in X_n \setminus \{ A \cup \{ \infty \} \}$$
where $A$ is the set of absorbing states in the original chain. (This $q_n$ is probably a really bad approximation of the actual absorption probability in the original chain if $A \not \subset X_n$, so you might want to assume that $A \subset X_1$, which will simplify the notation a little bit anyway.)
This follows from conditioning on the first step. This system can at least in principle be solved for any finite $n$, and one can then try to send $n \to \infty$ to obtain the solution to the original question through $q_n$. Ideally the system would be simple enough that you could do this analytically, or you could just approximate by taking a large enough finite $n$ and solving the system numerically.