Conditional Probability – Calculating Probabilities of an nth Step Transition Matrix for Discrete Time Markov Chains

conditional probabilitymarkov-processstochastic-processes

"Let $\{X_n, n \geq 0\}$ be a DTMC with state space $S = \{1, 2, 3, 4, 5\}$ and the following transition probability matrix:

$$
P = \begin{pmatrix} 0.1 & 0.0 & 0.2 & 0.3 & 0.4 \\ 0.0 & 0.6 & 0.0 & 0.4 & 0.0 \\ 0.2 & 0.0 & 0.0 & 0.4 & 0.4 \\ 0.0 & 0.4 & 0.0 & 0.5 & 0.1 \\ 0.6 & 0.0 & 0.3 & 0.1 & 0.0 \end{pmatrix}
$$

with the initial distribution as:

$$
a = \begin{pmatrix} 0.5 & 0 & 0 & 0 & 0.5 \end{pmatrix}.
$$

Compute $P(X_2 = 2, X_4 = 5)$"

Also, how do I do this? I'm not even sure what it means. Is this asking me to work out the probability of $X_4 = 5$ given that I have $X_2 = 2$?

EDIT: The matrix is correct. From the previous part of the question (that I didn't post), $P(X_2 = 2) = 0.08$. In the answers, my lecturer uses the Chapman Kolmogorov equations. Does this make the answer any clearer?

EDIT 2: Edited to bump to the top as I have commented on whuber's post with a couple of questions

Best Answer

Here is a graphical representation of the chain, with the vertices $2$ and $5$ highlighted:

figure 1

The initial state $a$ can be indicated by labeling the vertices with their values, highlighting the nonzero values:

figure 2

Two transitions from $a$, as computed by the matrix product $a\mathbb{P}\mathbb{P} = a\mathbb{P}^2$, is this distribution:

figure 3

The weight on vertex $2$ is precisely the chance of reaching $2$ after two steps; that is, it is $\Pr(X_2=2)$. To represent this event, we now zero out all other weights, leaving the distribution $b = (0, 2/25, 0, 0, 0)$. The question asks us to take two more steps, beginning at $b$, computing $b\mathbb{P}\mathbb{P} = b\mathbb{P}^2$:

figure 4

The labels give the new distribution. All two-step paths from vertex $2$ to vertex $4$ are highlighted. (There is just one, making it easy to compute the new distribution: $2/25$ is multiplied by $p_{2,4}=2/5$, giving $4/125$ for the transition from $2$ to $4$, then that is multiplied by $p_{4,5}=1/10$, yielding $4/1250=2/625$ for the double transition $2\to 4\to 5$. In more complicated situations we would have to examine all possible paths from $2$ to $5$ and add their contributions.)

Evidently, the chance of reaching vertex $2$ at step $2$ and then arriving at vertex $5$ at step $4$ is the final value at vertex $5$, $2/625 = 0.0032$.

Related Question