Difference between non-homogeneous Markov and Semi-Markov

Question

I have been going around with Markov family lately and now bit confused. As per my understanding, to satisfy Markov property, state holding time distribution needs to be exponential otherwise the process can’t satisfy the Markov property. The only way around this exponential distribution seemed to be semi-Markov processes (duration times can be arbitrary). In semi-Markov, state holding time modulates transition probabilities and in turn "time spent in a state affects the decision which state to enter next" (That means semi-Markov is not memory-less anymore?). Now non-homogeneous Markov processes have duration times that are not necessarily exponential. If that is the case that what are differences in general between non-homogeneous simple Markov and semi-Markov?

Answer 1

Let $X$ be a jump process on a (for simplicity) finite state space $S$. Denote with $\mathcal{F}^X$ the filtration generated by $X$, so that $\mathcal{F}_t^X$ is all the information generated by observing $X$ up until and including time $T$. Let $T(t)$ be the next jump time of $X$ after time $t$, i.e. $T(t) = \inf\{s \in (t,\infty) : X(s) \neq X(t)\}$.

To understand the differences between Markov and semi-Markov processes, the following suffices:

If $X$ is time-homogeneous Markov with transition rates $\mu_{jk}$, $j,k \in S, k \neq j$, then $T(t)-t$ given $\mathcal{F}^X_t$ is exponentially distributed with rate $\mu_{X(t)\bullet} := \sum_{k \neq X(t)} \mu_{X(t)k}$.

If $X$ is time-inhomogeneous Markov with transition rates $\mu_{jk}(t)$, now time-dependent, then $T(t)-t$ given $\mathcal{F}^X_t$ has density $s \mapsto \mu_{X(t)\bullet}(s) e^{-\int_t^s \mu_{X(t)\bullet}(u) \, \mathrm{d}u}$. (If the transition rates are time-independent, then this is the density of an exponential distribution, so we have a consistent extension of the time-homogeneous case.)

In the (time-homogeneous) semi-Markov case, we need the duration process $U(t) := t- \sup\{s \in [0,t] : Z(s) \neq Z(t)\}$, which is the time since the last jump of $X$. If $X$ is time-inhomogeneous Markov with transition rates $\mu_{jk}(u)$, then $T(t)-t$ given $\mathcal{F}_t^X$ has density $s \mapsto \mu_{X(t)\bullet}(U(t-)+s) e^{-\int_t^{t+s}\mu_{X(t)\bullet}(U(t-)+u-t) \, \mathrm{d}u}$, which depends on the current state $X_t$ as well as the current duration.

The time-inhomogeneous and semi-Markov case might look similar, but they are different. Consider the time interval $[0,2]$. Consider two cases. In both cases, we only have one jump. In the first case, the jump occurs at time $1$, and no further jumps occur up until time $2$. In the second case, a jump occurs at time $2$. We are interested in the distribution of the next jump time $T(2)$ minus 2. If $X$ is time-inhomogeneous Markov, we must in both cases consider the same density $\mu_{X(2)\bullet}(s) e^{-\int_2^s \mu_{X(2)\bullet}(u) \, \mathrm{d}u}$. If $X$ is semi-Markovian, the density in the first case is $\mu_{X(2)\bullet}(1+s) e^{-\int_0^s \mu_{X(2)\bullet}(1+u) \, \mathrm{d}u}$, while it is $\mu_{X(2)\bullet}(s) e^{-\int_0^s \mu_{X(2)\bullet}(u) \, \mathrm{d}u}$ in the second case.

Really, there are two 'clocks' in play. There is the 'time' clock, which ticks continuously throughout time, and then there is the 'duration' clock, which is reset at every jump. If you imagine that $X$ are health events of an individual, what is going one can be explained as follows. If $X$ is time-inhomogeneous Markov, the time until the next health event (healthy, ill, dead) is allowed to depend on the current health state of the individual (health, ill, dead) and his age (time). If $X$ is (time-homogeneous) semi-Markov, the time until the next health event is allowed to depend on the current health state of the individual and the time since he entered this state of health (duration). These two characteristics are of course very different.

Both aspects are combined in the time-inhomogeneous semi-Markov case with transition rates $\mu_{jk}(t,u)$ (that depend on time as well as duration). Here the time until the next health event is allowed to depend on the current health state of the individual, the time since he entered this state of health, and his sage.