[Math] Probability Markov chain, system of equations

linear algebramarkov chainsprobability theorysystems of equations

I'm looking for techniques or tricks to solve a system of linear equations you get where you want to find the limiting probabilities.

The system is this:

$\pi_0 = 0.7\pi_0 + 0.2\pi_1 + 0.1\pi_2$ ( EQUATION 2)

$\pi_1 = 0.2\pi_0 + 0.6\pi_1 + 0.4\pi_2$ (EQUATION 2)

$\pi_2 = 0.1\pi_0 + 0.2\pi_1 + 0.5\pi_2 $ (EQUATION 3)

And also:

$\pi_0 + \pi_1 + \pi_2 = 1$

What's an easy way to solve this?

I Reeeeeally don't want to put it into a matrix and do reduce row echelon form on it. Please tell me there is another way. I remember other folks solving it differently

EDIT:

I have figured it out. Maybe if one of yall are searching for answers on how to solve these Markov chains it will help.

First step is this:

$\pi_2 = 1 – \pi_0 – \pi_1$

Now I substitute this $\pi_2$ into equation 1 and 2 above.

For equation 1 I get:

$\pi_0 = 0.6\pi_0 + 0.1\pi_1 + 0.1$

For equation 2 I get:

$\pi_1 = -0.2\pi_0 + 0.2\pi_1 + 0.4$

For equation 1 and 2 we move the the $\pi_0$ and $\pi_1$ to the other sides of the equation setting both to zero:

$0 = -0.4\pi_0 + 0.1\pi_1 + 0.1$ EQUATION 1

$0 = -0.2\pi_0 – 0.8\pi_1 + 0.4$ EQUATION 2

Adding them together by multiplying constants can solve for both A and B. And once you know those 2 you can solve for C. I hope this was useful to anyone who has stumbled upon it

Best Answer

Here is an algorithm.

  • Use the normalizing condition $\pi_0 + \pi_1 + \pi_2 = 1$ to get rid of one unknown, say $\pi_2$, in two equations, say equations (1) and (2).
  • This yields an affine system of two equations in the unknowns $(\pi_0,\pi_1)$. Compute the Cramer determinant of this system.
  • If the Cramer determinant is not zero, solve for $(\pi_0,\pi_1)$ using Cramer formulas.
  • Remember that $\pi_2=1-\pi_0 - \pi_1 $ to deduce $\pi_2$. You are done.
  • If the Cramer determinant is zero, apply the same steps to two other equations, say equations (2) and (3). Then the Cramer determinant will be nonzero (unless several stationary distributions exist because the chain is reducible) hence, this time, you will carry through.

This applies to $n\times n$ systems to compute stationary distributions on state spaces of size $n$, for every $n$.

Related Question