If I am not mistaken, the steady state distribution is independent of initial state distribution, and regular Markov chains satisfies this definition.
On the other hand, since the row of each limiting matrix for an absorbing Markov chain is the same, the state distribution after a large number of transitions for an absorbing Markov chain is dependent on the initial state distribution.
However, I read somewhere that an absorbing Markov chain can have steady state distributions – which contradicts what I have always believed. I mean, I know what they are trying to imply (as in the chain will converge to a fixed probability), but is it the correct term?
Best Answer
Let $X=\{X_n:n\in\mathbb N_0\}$ be a Markov chain with state space $S$ and transition probabilities $p_{ij}$. A vector $\pi\in\mathbb R^{\#S}$ is a stationary distribution for $X$ iff
For each state $i\in S$, let $C_i = \{j\in S : i\leftrightarrow j\}$ be the communicating class of $i$. Assume that each class is positive recurrent, that is, for each $i\in S$, the return time $$\tau_{ii} = \inf\{n>0: X_n = i\mid X_0=i\} $$ has finite expectation. Then there exists a unique stationary distribution $\pi^{(i)}$ for $X$ conditioned on $X_0\in C_i$, with $$\pi^{(i)}_j = \frac1{\mathbb E[\tau_{jj}]},\ j\in C_i. $$ Let $\bar\pi^{(i)}$ be the extension of $\pi^{(i)}$ to $S$ with $\bar\pi^{(i)}_j=0$ for $j\in S\setminus C_i$, then the set of stationary distributions for $X$ is the convex hull of $\{\bar\pi^{(i)}:i\in S\}$. If there is a class $C_i$ which is not positive recurrent, then no stationary distribution exists for that class, and hence no stationary distribution exists for $X$.
In the case where $X$ is absorbing, however, the communicating classes are simply the singletons $\{i\}$ for the absorbing states with $p_{ii}=1$. Let $A\subset S$ be the set of absorbing states, then $\pi$ is a stationary distribution iff $\pi_j=0$ for $j\in S\setminus A$.