Solved – When is Markov chain a generator for iid sequences

markov-processrandom variableself-study

Definition: Markov chain as : A stochastic process $X_1, X_2, \ldots, $ is a Markov process ( or Markov chain) is for any discrete time index $n = 1,2,\ldots,$ , $ Pr(X_{n+1} = x_{n+1}| X_n = x_n, \ldots, X_1 = x_1) = Pr(X_{n+1} = x_{n+1}| X_n = x_n)$.

Q1: What is the meaning of the above statement in English?

Q2: Is Markov chain composed of an independent and identically (iid) random variables. Or when the Markov chain is stationary, then we can say that it is iid random variables?

Thank you

Best Answer

Q1: The above statement means that the probability of a random variable X being equal to some value x at time n + 1, given all the x values that came before it in the sequence, is equal to the probability of X being equal to some value x at time n + 1 given just the value of x that came before it. In other words, X at time n + 1 is only dependent on x at time n, not any other value of x. So in a sequence, you can say that X at time n + 1 is independent of all other x except X at time n.

Q2: By the answer to Q1, all values in the Markov Chain are not independent of each other because $P(X_{n+1} | X_n) \ne P(X_{n+1})$. After enough iterations, the chain (usually) converges in distribution so they would be identically distributed.

That's all I know.

Related Question