Suppose you have the following Markov Chain {$X_n$} $n=0,1,2,3$ with transition probability matrix:
$P=\begin{bmatrix}1 & 0&0&0\\P_{10} &P_{11}&P_{12}&P_{13}\\P_{20}&P_{21} &P_{22}&P_{23}\\0&0&0&1\end{bmatrix}$
So clearly 0 and 3 are absorption states while 1 and 2 are transient.
If I define
$T=min(n\ge0;X_n=0$ or $X_n=3$)
$P_{ij}=P(X_{n+1}=j|X_n=i)$ probability to go from i to j in one step.
$u_i=P(X_T=0|X_0=i)$ $u_i$ is the probability of absorption in state 0 starting from state i.
$v_i=E[T|X_0=i]$ $v_i$ is expected time until absorption.
Could someone explain/show the MATHEMATICAL developments starting from the formula $v_i=E[T|X_0=i]$ and ending up with the equations:
$v_1=1+P_{11}v_1+P_{12}v_2$
$v_2=1+P_{21}v_1+P_{22}v_2$
it would be fine to just show how to do $v_1$ or $v_2$, either one.
I am not super interested in whatever "intuitiv explanation" of how you get these equations for $v_1$ and $v_2$, I am more interested in the mathematical developments starting from just the formula for $v_i$ given above and showing all the steps and how they end up with said equations.
Obviously I understand that $v_0=v_3=0$ has to be since 0 and 3 are absorption states and that a process can never "leave" once its absorbed.
In a similar way I am also interested to understand how the equations for the u's make sense mathematically.
$u_1=P_{10}+P_{11}u_1+P_{12}u_2$
$u_2=P_{20}+P_{21}u_1+P_{22}u_2$
starting from just $u_i=P(X_T=0|X_0=i)$
Here as well I understand that the probability to get absorped in 0 if you start in 3 is 0 so $u_3=P(X_T=0|X_0=3)=0$ and that the probabitliy to get absorped in 0 starting from 0 is 1 so $u_0=P(X_T=0|X_0=0)=1$
So the equation for $u_1$
is given by $u_1=P_{10}u_0+P_{11}u_1+P_{12}u_2+P_{13}u_3$
or written in another form
$P(X_T=0|X_0=1)=P(X_1=1|X_0=0)P(X_T=0|X_0=0)+P(X_1=1|X_0=1)P(X_T=0|X_0=1)+P(X_1=1|X_0=2)P(X_T=0|X_0=2)+P(X_1=1|X_0=3)P(X_T=0|X_0=3)$
why exactly is the left hand side and right hand side equal? is this just the law of total probability or whatever it is called?
Best Answer
For the first question, for $i=1$ we have
$E[T|X_0=1]= E[T|X_0=1,X_1=1]P_{ 11}+ E[T|X_0=1,X_1=2]P_{ 12}+ E[T|X_0=1,X_1=3or0](P_{ 10}+P_{13}) $
Which by the Markov property is (and doing a plus minus one at the same time)
$E[T|X_0=1]= E[(T-1)+1|X_1=1]P_{ 11}+ E[(T-1)+1|X_1=2]P_{ 12}+ E[(T-1)+1|X_1=3or0]( P_{ 10}+P_{13}) = E[(T-1)|X_1=1]P_{ 11}+ E[(T-1)|X_1=2]P_{ 12}+ E[(T-1)|X_1=3or0]( P_{ 10}+P_{13})+( P_{ 11} +P_{ 12} +P_{ 10} +P_{ 13}),$
where the last bracket equals 1.
Then, since we have a time homogenous Markov chain,
$E[(T-1)|X_1=1] = E[T|X_0=1] $.
(Similarily for the other expectations).
The other equations for $u_i$ and $v_i$ follow similarily. Total probability law is the right intuition.