[Math] Markov Chain with two states

linear algebramarkov chains

A Markov Chain has two states, $A$ and $B$, and the following probabilities: If it starts at $A$, it stays at $A$ with probability $\frac13$ and moves to $B$ with probability $\frac23$; if it starts at $B$, it stays at $B$ with probability $\frac15$ and moves to $A$ with probability $\frac45$. Let $X_n$ denote the state of the process at step $n$, $n=0,1,2,…$. If the process is equally likely to start in each state at time $0$, find the probability that $X_1 = X_2$.

I know that the transition matrix looks like this: $ P = [\frac13 , \frac23; \frac15 , \frac45]$ but I am not sure what to do next.

Please help 🙂


If someone could check me, I got a final answer of $0.504741$.

Best Answer

For starters, you have the wrong transition matrix. You want $P$ to be such that

$$P{1\brack 0}={\frac{1}{3}\brack \frac{2}{3}}\;,$$

meaning that if you start with something in-state $A$, after one transition it’s still in state $A$ with probability $\frac13$ and in-state $B$ with probability $\frac23$. Similarly, you want

$$P{0\brack 1}={4/5\brack 1/5}\;,$$

meaning that if you start with something in state $B$, after one transition it’s still in state $B$ with probability $\frac15$ and has moved over to state $A$ with probability $\frac45$. This means that you want

$$P=\begin{bmatrix}1/3&4/5\\2/3&1/5\end{bmatrix}\;.$$

For $n\in\Bbb N$ let $B_n$ be the $2\times 1$ matrix whose top entry is the probability that $X_n$ is $A$, and whose bottom entry is the probability that $X_n$ is $B$. You’re told that $$B_0={1/2\brack1/2}\;:$$ the process is equally likely to start in each of the states, so initially it’s in each state with probability $\frac12$. You also know that $B_{n+1}=PB_n$ for each $n\in\Bbb N$: that’s simply how the Markov process works. Thus,

$$B_1=\begin{bmatrix}1/3&4/5\\2/3&1/5\end{bmatrix}B_0=\begin{bmatrix}1/3&4/5\\2/3&1/5\end{bmatrix}\begin{bmatrix}1/2\\1/2\end{bmatrix}=\begin{bmatrix}17/30\\13/30\end{bmatrix}\;,$$

where I’ll leave the conclusion of the calculation to you.

Once you have this, you need to find the probability that $X_1=X_2$. That’s the probability that

$$(X_1=A\text{ and }X_2=A)\quad\text{or}\quad(X_1=B\text{ and }X_2=B)\;;$$

can you find that? Remember what the transition probabilities mean.