(Transient) Birth and Death Process

markov chainsstochastic-processes

Consider a chain $X_t$ with birth rates $\lambda_n = n \lambda$ for some $\lambda$, death rates $\mu_n = \frac{n}{2}$ for $n \geq 1$ where n is the current population.

I am asked the following question:

Given the chain is transient and that $X_0=4$, what is the probability that the chain never reaches 0?

Some thoughts: If the chain is transient, then we must have $\lambda > \frac{1}{2}$. Suppose instead that $\lambda_0 = 0$ (aka 0 is an absorbing state), then we must find $P(X_t > 0 \forall t>0)=1-P(X_t=0, \text{for some t})$.

Best Answer

The solution is the following, for those interested:

Let $\lambda_0=0$, as we only care about the first return to $0$. This makes $0$ an absorbing state. Let $a(n)$ denote the probability that a population will ever reach 0, given that it started with $X_0=n$. Then we have the following: $$a(n)=\frac{\lambda_{n}}{\lambda_{n}+\mu_{n}}a(n+1)+\frac{\mu_{n}}{\lambda_{n}+\mu_{n}}a(n-1)$$

Recursively, this can be written as $$a(n+1)=(a(1)-1)\sum_{j=0}^n\frac{\mu_1 \mu_2 \cdots \mu_j}{\lambda_1 \lambda_2 \cdots \lambda_j}+1 \quad (*)$$

Thus we are interested in finding $a(4)$, the probability that we will ever reach zero given we started with $X_0=4$.

To find $a(1)$, we let $n \to \infty$ in (*). We are allowed to do this because we are given that the chain is transient. In other words it has a non-zero probability of escaping to infinity. Thus we have $$0 = 1+[a(1)-1]\sum_{j=0}^\infty\frac{\mu_1 \mu_2 \cdots \mu_j}{\lambda_1 \lambda_2 \cdots \lambda_j}$$

The left-hand term is $0$ because $\lim_{n \to \infty} a(n) = 0$, as if we started with an increasingly large population, the probability that it will die out is $0$. Now let's take a look at the following term:

$$\frac{\mu_1 \mu_2 \cdots \mu_n}{\lambda_1 \lambda_2 \cdots \lambda_n}=\frac{n!}{2^n} \cdot \frac{1}{n! \cdot \lambda^n}=(\frac{1}{2\lambda})^n$$

Then $$\sum_{j=0}^\infty\frac{\mu_1 \mu_2 \cdots \mu_j}{\lambda_1 \lambda_2 \cdots \lambda_j}=\sum_{j=0}^\infty(\frac{1}{2\lambda})^j$$ This sum converges if and only if $\frac{1}{2\lambda}$<1, which we are given from transience.

Thus $$a(1)=\frac{\sum_{j=0}^\infty(\frac{1}{2\lambda})^j-1}{\sum_{j=0}^\infty(\frac{1}{2\lambda})^j}=\frac{\frac{1}{1-\frac{1}{2\lambda}}-1}{\frac{1}{1-\frac{1}{2\lambda}}}=\frac{1}{2\lambda}$$

Then $$a(4)=(a(1)-1)\sum_{j=0}^3\frac{\mu_1 \mu_2 \cdots \mu_j}{\lambda_1 \lambda_2 \cdots \lambda_j}+1=(\frac{1}{2\lambda}-1)\sum_{j=0}^3(\frac{1}{2\lambda})^j+1=\frac{1}{(2\lambda)^4}$$

Thus the probability that this chain never reaches zero, given that $X_0=4$, is just $1-a(4)=1-\frac{1}{(2\lambda)^4}$.

Related Question