[Math] Eigenvector of a markov chain – relation to initial conditions

eigenvalues-eigenvectorslinear algebramarkov chainsstochastic-processes

Something I'm a bit confused about when it comes to Markov chains is how the initial vector relates to the eigenvector. Suppose the transformation matrix is $\vec{A}$. From my understanding, we are looking to find a vector $\vec{v}$ which has the equality:

$$\vec{v}*\vec{A}=\vec{v}$$

In other words (and please correct me if I'm wrong), we are trying to find the eigenvector of $\vec{A}$ with a corresponding eigenvalue of 1. However, this is not dependent on initial conditions $\vec{v_0}$ – I get how the iterative approach will get to a stable equilibrium (and uses the initial vector as the starting point), but with exact solutions, how does the eigenvector then take us from our initial condition vector to the stable equilibrium vector?

Best Answer

Consider the iterations $$v_{i+1}^T=v_i^TA$$

If $v_i$ indicates the probability distribution of states at time $i$, then $v_{i+1}$ indicate the distribution of states at time $i+1$.

In fact, convergence is not guaraneteed. For example, consider $\begin{bmatrix} 0 & 1 \\ 1 & 0\end{bmatrix}$, suppose we let $v_0=\begin{bmatrix} 0.3 \\ 0.7\end{bmatrix}$, we can observe that the sequence of vectors will just iterate between $\begin{bmatrix} 0.3 \\ 0.7\end{bmatrix}$ and $\begin{bmatrix} 0.7 \\ 0.3\end{bmatrix}$ and hence it doesn't conveges. This is true for almost all the vectors besides $v_0 = \begin{bmatrix} 0.5 \\ 0.5\end{bmatrix}$.

Hence, the initial distribution does decide whether it converges and if so to which equilibrium distribution. There are also cases where equilibrium solution is unique and it is independent of initial distribution.

Short answer when things are nice:

Let's consider the special case where $\lim_{n \rightarrow \infty} A^n =B$ exists.

Notice that $B$ has the property that $BA=B$.

Let's consider the vector $y^T=v_0^TB$.

Then, $y^TA = v_0^TBA=v_0^TB=y^T$

Hence $y$ is an eigenvector. Each row of $B$ is a distribution and $v_0$ decides the weight for combinations of such distributions. In the events that each row of $B$ is identical, then it is independent of the choice of $v_0$. Ergodic unichain guarantees that each row of $B$ is identical.