[Math] Understanding the “first step analysis” of absorbing Markov chains

probability theorystochastic-processes

Consider a time-homogeneous Markov chain $\{X_n\}_{n=0}^\infty$ with the state space state space $S=\{0,1,2\}$ and the following transition probability matrix:
\begin{pmatrix}
1 & 0 & 0 \\
\alpha & \beta & \gamma \\
0 & 0 & 1
\end{pmatrix}
where $\alpha,\beta,\gamma>0$. Note that state 0 and 2 are absorbing.

Let
$
T=\min\{n\geq 0\mid X_n=0\textrm{ or }X_n=2\}
$
be the time of absorption of the process. It is intuitively true that
$$
P(X_T=0\mid X_1=1)=P(X_T=0\mid X_0=1)\tag{*}
$$
which is the key point of the so called "first step analysis". See for instance Chapter 3 in Karlin and Pinsky's Introduction to Stochastic Modeling. But the book does not bother giving a proof of it.

Here is my question:

How can one prove (*) using the definition of conditional probability and the Markov property?

Best Answer

Here is a formal proof. For any sequence ${\bf x}=(x_0,x_1,x_2,\dots)$ in $\{0,1,2\}^{\mathbb{N}}$ define $$H({\bf x})=\sum_{n=0}^\infty \,\prod_{j=0}^{n-1} {\bf 1}[x_j=1]\,{\bf 1}[x_n=0],$$ so that $H({\bf x})=1$ if the sequence $\bf x$ hits "0" before hitting "2", $H({\bf x})=0$ otherwise. Note that an initial value of "1" can be dropped, i.e., $$H(1,x_1,x_2,\dots )=H(x_1,x_2,\dots).\tag1$$

Since $\mathbb{P}(X_0=1\mid X_1=1)=1$, under the measure $\mathbb{P}(\,\cdot \mid X_1=1)$ equation (1) gives
$$H(X_0(\omega), X_1(\omega),\dots)=H(X_1(\omega), X_2(\omega),\dots),$$ almost surely. Therefore, $$E(H(X_0,X_1,\dots)\mid X_1=1)= E(H(X_1,X_2,\dots)\mid X_1=1)= E(H(X_0,X_1,\dots)\mid X_0=1),$$ where the last equation follows since $(X_n)$ is a time homogeneous Markov process.

That is, $$P(X_T=0, T<\infty\mid X_1=1)=P(X_T=0, T<\infty\mid X_0=1).$$

Related Question