General Two-State Continuous Markov Chain – Transition Probability Matrix not Valid.

eigenvalues-eigenvectorslinear algebramarkov chainsordinary differential equationstransition matrix

We have the following transition rate matrix for a two-state Markov Chain:
$$
Q = \begin{pmatrix}
-\lambda_1 & \lambda_2 \\
\lambda_1 & -\lambda_2
\end{pmatrix}
$$

Note I use the convention where the columns add to zero. I want to find the transition probability matrix P(t) using the differential equation:
$$
\frac{dP(t)}{dt} = Q P(t) \ .
$$

We can find $P(t)$ by letting $P(0)=I$ the identity matrix such that we get:
$$
P(t) = \exp(Qt) \ .
$$

By writing $Q = U D U^{-1}$ where $D$ is a diagonal matrix consisting of eigenvalues of $Q$ and $U $ is a matrix consisting of eigenvectors of $Q$. The eigenvalues of $Q$ are $0$ and $-(\lambda_1 + \lambda_2)$ and the eigenvectors are $(\lambda_2 , \lambda_1)^T$ and $(1 , -1)^T$. Then we get the following:
\begin{align*}
D &= \begin{pmatrix}
0 & 0 \\
0 & -(\lambda_1 + \lambda_2)
\end{pmatrix} \\[10pt]
U &= \begin{pmatrix}
\lambda_2 & 1 \\
\lambda_1 & -1
\end{pmatrix} \\[10pt]
U^{-1} &= \frac{1}{\lambda_1+\lambda_2}\begin{pmatrix}
1 & 1 \\
\lambda_1 & -\lambda_2
\end{pmatrix} \ .
\end{align*}

Lets check that $Q = U D U^{-1}$ is correct:
\begin{align*}
U D U^{-1} &= \begin{pmatrix}
\lambda_2 & 1 \\
\lambda_1 & -1
\end{pmatrix} \begin{pmatrix}
0 & 0 \\
0 & -(\lambda_1 + \lambda_2)
\end{pmatrix} \frac{1}{\lambda_1+\lambda_2}\begin{pmatrix}
1 & 1 \\
\lambda_1 & -\lambda_2
\end{pmatrix} \\[10pt]
&= \begin{pmatrix}
\lambda_2 & 1 \\
\lambda_1 & -1
\end{pmatrix}
\begin{pmatrix}
0 & 0 \\
-\lambda_1 & \lambda_2
\end{pmatrix} \\[10pt]
&= \begin{pmatrix}
-\lambda_1 & \lambda_2 \\
\lambda_1 & -\lambda_2
\end{pmatrix} \\[10pt]
&= Q \ .
\end{align*}

Now we calculate $P(t) = \exp(Qt)$ using $\exp(UDU^{-1}t) = U \exp(Dt)U^{-1}$ then we get:
\begin{align*}
P(t) &= U \exp(Dt)U^{-1} \\[10pt]
&= \begin{pmatrix}
\lambda_2 & 1 \\
\lambda_1 & -1
\end{pmatrix} \begin{pmatrix}
0 & 0 \\
0 & \exp(-(\lambda_1 + \lambda_2)t)
\end{pmatrix} \frac{1}{\lambda_1+\lambda_2}\begin{pmatrix}
1 & 1 \\
\lambda_1 & -\lambda_2
\end{pmatrix} \\[10pt]
&= \frac{\exp(-(\lambda_1 + \lambda_2)t)}{\lambda_1+\lambda_2} \begin{pmatrix}
\lambda_2 & 1 \\
\lambda_1 & -1
\end{pmatrix} \begin{pmatrix}
0 & 0 \\
0 & 1
\end{pmatrix} \begin{pmatrix}
1 & 1 \\
\lambda_1 & -\lambda_2
\end{pmatrix} \\[10pt]
&= \frac{\exp(-(\lambda_1 + \lambda_2)t)}{\lambda_1+\lambda_2} \begin{pmatrix}
\lambda_2 & 1 \\
\lambda_1 & -1
\end{pmatrix}
\begin{pmatrix}
0 & 0 \\
\lambda_1 & -\lambda_2
\end{pmatrix}\\[10pt]
&= \frac{\exp(-(\lambda_1 + \lambda_2)t)}{\lambda_1+\lambda_2}
\begin{pmatrix}
\lambda_1 & -\lambda_2 \\
-\lambda_1 & \lambda_2
\end{pmatrix} \ .
\end{align*}

The problem with this results is that the columns do not add to one and therefore this is not a valid transition probability matrix. What have I done wrong?

Best Answer

Most of your work is right. The only mistake is that you should have $$ \exp(Dt) = \begin{pmatrix} \color{red}1 & 0 \\ 0 & \exp(-(\lambda_1 + \lambda_2)t) \end{pmatrix}. $$ In the end, this means that you'll have to add $$ \frac 1{\lambda_1 + \lambda_2} \begin{pmatrix} \lambda_2 & 1 \\ \lambda_1 & -1 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix} \begin{pmatrix} 1 & 1 \\ \lambda_1 & -\lambda_2 \end{pmatrix} = \frac 1{\lambda_1 + \lambda_2} \pmatrix{\lambda_2 & \lambda_2\\ \lambda_1 & \lambda_1} $$ to your final answer.

Related Question