Discretization of a Continuous Time Markov Chain

markov chainsmarkov-processprobability theorystochastic-processes

Let $(X_t)_t$ be a continuous time markov chain on a finite state space with initial distribution $\alpha$ and transition matrix $A$.

Suppose now we only observe $X$ at certain discrete time steps $n \cdot t$ for some $t > 0$, i.e. we define $Y_n=X_{n\cdot t}$ and $Y_0$ is distributed as $\alpha$. I would expect that $Y_n$ becomes a discrete time markov chain, and that its transition matrix (say $B$) can be derived from $A$.

Best Answer

Since the process $X_s$ for $s\ge t$ only depends on the state at $X_t$, not before time $t$, the state at discrete times $X_{kt}$ for integers $k\ge n$ only depends on the state $X_{nt}$ and not before, which is the definition of a discrete time Markov process.

If $A_{ij}$ is the transition intensity $i\rightarrow j$, define the vector $a$ by $a_i=\sum_j A_{ij}$ and the matrix $Q_{ij}=A-D_{a}=a_{ij}-a_i\delta_{ij}$.

Now, the transition probability $P_{tij}$ of $i\rightarrow j$ over a time period $t$ is $$ P_t = e^{tQ} = \lim_{n\rightarrow\infty}\left(I+\frac{tQ}{n}\right)^n = I + tQ + \frac{t^2Q^2}{2!} + \cdots. $$