Probability of a sum of two die rolls being 5 and not 7 in a game

Question

I've been working on this problem for a while and I'm not sure how to approach it. This was given in a class on Stochastic Processes, and it's meant to be solved using a Markov chain:

A single die is rolled repeatedly. The game stops the first time that the sum of two successive rolls is either 5 or 7. What is the probability that the game stops at a sum of 5?

What's the correct way to tackle this problem?

Answer 1

You can obviously do this with nine states: the start, six transitional states depending on the previous roll, and the two end states. You can reduce these to six states:

• start
• previous roll 1 or 2 but not stopped yet
• previous roll 3 or 4 but not stopped yet
• previous roll 5 or 6 but not stopped yet
• stopped at sum of 5
• stopped at sum of 7

and a transition matrix of

\begin{pmatrix} 0 & \frac26 & \frac26 & \frac26 & 0 & 0 \\ 0 & \frac26 & \frac16 & \frac16 & \frac16 & \frac16 \\ 0 & \frac16 & \frac16 & \frac26 & \frac16 & \frac16 \\ 0 & \frac16 & \frac26 & \frac26 & 0 & \frac16 \\ 0 & 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 \\ \end{pmatrix}