[Math] What does a steady state vector tell us if the Markov chain is irregular

linear algebramarkov chainsstochastic-processes

When a Markov chain is regular, the finding the steady state vector (i.e. the eigenvector corresponding to the eigenvalue $1$) will tell us the long term probability of ending up in any of the states, regardless of which state you started in.

However, if the Markov chain is irregular, this no longer holds, as which state you begin in, as well as the number of transitions, will determine which state you end up in the long run.

I was under the impression though, that a steady-state vector is still a concept even when referring to irregular Markov Chains. Additionally, Markov chains appear to always have an eigenvalue of $1$, whether or not they are regular. My question is, what does a steady-state vector of an irregular Markov chain tell us?

Best Answer

Note that two things can go wrong for irregular matrices.

One thing that can go wrong is that the matrix is reducible. What this means for the Markov chain is that there is a non-zero possibility of staying within a proper subsets of the available states. In such a case, we'll end up with a space of steady-state vectors, corresponding to multiple "final distributions".

Another thing that can go wrong is that the matrix is periodic. What this means is that you're caught in cycles of a particular length, so that you can only get back to where you started on steps that are multiples of some period $n$. In such a case, you get most of the same properties that you had before, only now you'll have multiple (complex) eigenvalues of absolute value $1$. There is still, however, a unique steady-state vector.

It turns out that these are the only things that can go wrong, in some combination. For more information on this and everything I've said here, see this wiki page and, in particular, the Frobenius theorem for irreducible matrices.