[Math] How to find transition probability matrix $P$ by using transition rate matrix $T$

markov chainsprobability theory

Let
$$T = \left(\begin{matrix} -2 & 1 & 1&0 \\
2 & -3 & 1&0 \\
1 & 2 & -4 & 1\\
1 & 3 & 1 & -5\end{matrix} \right) $$

be a transition rate matrix of an imbedded discrete time Markov chain with state space $S=\{0,1,2,3\}$ of a continuous time Markov chain $X$ . Then how do we get the transition probability matrix $P$ of $X$ by using $T$?

Best Answer

Before answering the question, it seems necessary to dispel some misconceptions:

  • There is no such thing as "the" transition matrix of a continuous time Markov process but a semi-group $(P_t)$ such that each $P_t$ is the transition matrix from $X_s$ to $X_{s+t}$, for every $s$.
  • The matrix $T$ is not a transition matrix since it has some negative entries.
  • The matrix $T$ is not a transition rate matrix of an imbedded discrete time Markov chain since $T_{03}=0\ne T_{02}T_{23}$.

The matrix $T$ is the generator of a continuous time Markov chain hence, by definition, $$P_t=\mathrm e^{tT}=\sum_{n\geqslant0}\frac{t^n}{n!}T^n.$$ A general approach to compute each $P_t$ is to diagonalize the symmetric matrix $T$ as $T=P^{-1}DP$ for some diagonal matrix $D$ with entries $\lambda_i$ the eigenvalues of $T$, then $\mathrm e^{tD}$ is diagonal with entries $\mathrm e^{t\lambda_i}$ and $$P_t=\mathrm e^{tT}=P^{-1}\mathrm e^{tD}P.$$

Related Question