Full Problem: A continuous-time Markov chain has generator matrix
$$Q=
\begin{pmatrix}
-1 & 1 & 0 \\
1 & -2 & 1 \\
2 & 2 & -4 \\
\end{pmatrix}
$$
(i) Exhibit the transition matrix of the embedded Markov chain.
(ii) Exhibit the holding time parameters for each state.
OK, I must be misunderstanding something. I have the following for the embedded chain transition probabilities:
$$P_{ij} = \frac{q_{ij}}{q_i}$$
where $q_i$ is the parameters of the exponential length of time that the process stays in i.
The problem is that I know $q_i$ is the minimum with exponential distribution and parameter $\sum_k q_{ik}$.
But when I calculate $P_{11}$ that gives me $-1$ which is impossible..
Obviously there is an error in my thinking but can anyone shine light on what I'm misunderstanding? Thank you greatly in advance.
Best Answer
Indeed $P_{ij} = \frac{q_{ij}}{q_j}$ for $i\ne j$, so the transition matrix for the embedded Markov chain is $$ P=\pmatrix{ 0&1&0\\ \frac12&0&\frac12\\ \frac12&\frac12&0 }. $$ The holding time for state $0$ has $\mathsf{Exp}(1)$ distribution since $q_{01}=1$ and $q_{02}=2$. The holding time for state $1$ has $\mathsf{Exp}(2)$ since $q_{10}=q_{12}=1$ and the minimum of two independent exponential random variables is again exponential with rate equal to the sum of the two. A similar argument shows that the holding time for state $2$ has $\mathsf{Exp}(4)$ distribution.