I am trying to explain the differences between the following concepts to someone and I realized I myself am super confused:
Continuous/discrete Markov Process
Continuous/Discrete Markov chains
Markov property : $\mathrm{P}\{X_n=i|X_{n−1}=j,X_{n−2}=k,…\}=\mathrm{P}\{X_n=i|X_{n−1}=j\}\mathrm{P}\{X_n=i|X_{n−1}=j,X_{n−2}=k,…\}=\mathrm{P}\{X_n=i|X_{n−1}=j\} ?$
I used to think: Every process that has Markov property is a Markov Process. Every Markov process is a Markov chain and every Markov chain is a Markov process.
But it seems crazy now when I think about it, because if they are all the same, why there are different names for them?
And they are continuous (discrete) if their parameter set TT is continuous (discrete) regardless of their state space?
I want to start with homogeneous Markov chain and process too. But since I am already too confused and Wikipedia is making me more confused, I prefer to wait till I get these basic definitions straight first (any nice analogy that can be useful to teach them to others would be highly appreciated too if any teacher here knows any.).
Thanks a lot
Best Answer
It might be important to differentiate between the various stochastic process types based on both state space and time variable. (Note: discrete space/time can also be called countable.) So there are 4 types:
The next part is not so clearly agreed upon in the literature. I will simply state the definitions I am used to seeing.
A Markov process is any stochastic process that satisfies the Markov property. It doesn't matter which of the 4 process types it is.
A Markov chain is a Markov process with a discrete state space (i.e. can be type 1 or 3).
A Discrete-time Markov chain (or discrete Markov chain) is a Markov process in discrete time with a discrete state space (i.e. type 1, above).
A Continuous-time Markov chain (or continuous Markov chain) is a Markov process with a discrete state space in continuous time (i.e. of type 3). (E.g. see here).
A Stationary process is a stochastic process with a joint probability distribution that does not change when translated in time (see here).
A Time-homogeneous Markov chain is a stationary Markov chain. This means that the transition probabilities do not change in time. So, the probability of going from one state $s_1$ to another state $s_2$, once you are at $s_1$, is always the same (i.e. it doesn't matter when you get there).
A discrete-time stationary Markov chain is the most classic case (and in fact what most people mean when they say Markov chain).