[Math] Calculating transition probabilities

markov chainsprobability

I'm confused about a certain theory and I need some advice

789

456

123

This grid represents a basic house with 9 rooms.

A mouse, m, is in room i, and after a certain time interval, "t", it moves to an adjacent room, j. For example, the mouse can be in room 1, and after t, will either not move, or move to either room 2, or 4. Let's say it moved to room 2. After another "t" interval, it will either move to 1/3/5 or stay in 2. And so on. So it's a markov chain.

What method do I use to calculate the transition probabilities between each state where the rules above are all valid and the resulting probability of, after a long time, being in the bottom row (1/2/3) is 1/6, being in the middle 3 row (4/5/6) is 2/6 (1/3), and the top row (7/8/9) is 3/6 (1/2)?

Best Answer

I said in comments that I thought you do not have information from the long term distribution about moving left or right, and only partial information about moving up or down. But you can say that the transition probability of moving from the bottom to the middle row is double the transition probability of moving from the middle row to the bottom row, while the transition probability of moving from the middle to the top row is $1.5$ times the transition probability of moving from the top row to the middle row

I am still not clear about the question, but let's suppose any answer meeting the condition will do, so then you could have for example

  • $\Pr(1 \to 2)= \Pr(1 \to 4) = \Pr(2 \to 1)= \Pr(2\to 3)=\Pr (2 \to 5) = \Pr(3 \to 2)$ $=\Pr(3 \to 6) = 0.3$
  • $\Pr(4 \to 1)= \Pr(4 \to 5) = \Pr(4 \to 7)= \Pr(5\to 2)=\Pr (5 \to 4) = \Pr(5 \to 6)$ $=\Pr(5 \to 8) =\Pr(6 \to 3) =\Pr(6 \to 5) =\Pr(6 \to 9) = 0.15$
  • $\Pr(7 \to 4)= \Pr(7 \to 8) = \Pr(8 \to 5)= \Pr(8\to 7)=\Pr (8 \to 9) = \Pr(9 \to 6)$ $=\Pr(9 \to 8) = 0.1$

implying probabilities of no movement in a particular time step of

  • $\Pr(1 \to 1) = 0.4$, $\Pr(2 \to 2) = 0.1$, $\Pr(3 \to 3) = 0.4$, $\Pr(4 \to 4) = 0.55$, $\Pr(5 \to 5) = 0.4$, $\Pr(6 \to 6) = 0.55$, $\Pr(7 \to 7) = 0.8$, $\Pr(8 \to 8) = 0.7$, $\Pr(9 \to 9) = 0.8$

If you simulate this with any starting position, I would expect that after say $100$ steps you would find the probability of each of the positions $1$ to $3$ having probability close to $\frac1{18}$, of each of the positions $4$ to $6$ having probability close to $\frac1{9}$, and of each of the positions $7$ to $9$ having probability close to $\frac1{6}$, adding up by row to $\frac16$, $\frac13$ and $\frac12$, which is what the question asked for