Solved – Aperiodicity in markov chain

markov-process

given this transition matrix of markov chain

1/2 1/4 1/4
0   1/2 1/2
1   0    0    

which represents transition matrix of states a,b,c.
a has probability of 1/2 to itself 1/4 to b 1/4 to c.
b has probability 1/2 to itself and 1/2 to c
c has probability 1 to a.

1) why is state c aperiodic?

I know that it is irreducible and state a is aperiodic because it has self loop
so all states are aperiodic. but i can't see why states that don't have self loops are aperiodic.

if one can explain what exactly aperdicity and why state c is aperiodic from the definition of aperiodicity itself.

Best Answer

Definition Let $p_{ii}^{(n)}$ denote the probability of returning to state $i$ at step $n$ and let $t\in\{2, 3\dots\}$. State $i$ is said to be periodic with period $t$ iff

  • $ p_{ii}^{(n)} = 0 $ for $n \neq t, 2t, \dots$
  • $ p_{ii}^{(n)} \neq 0 $ for $n = t, 2t, \dots$

If we can not find a $t$ such that this holds, the state is said to be aperiodic.

Solution In your case it would be useful to draw a transition diagram of the matrix. You can see that if the chain starts in $c$ then returns to $c$ are possible at steps $2, 3, 4, 5, \dots$. As we can not find a $t$ such that the definition holds, $c$ is an aperiodic state.

╔═════╦═════╗
║  n  ║  p  ║
╠═════╬═════╣
║ 1   ║ 0   ║
║ 2   ║ >0  ║
║ 3   ║ >0  ║
║ ... ║ ... ║
╚═════╩═════╝
Related Question