[Math] Gambler’s ruin and Markov Chain, coin toss and stakes

gamblingmarkov chainsmarkov-processprobability

I'm considering a classical problem about Markov Chains:

A gambler has $£8$ and wishes to get to $£10$. A coin is tossed repeatedly : if it comes down tails, the gambler loses his stake, and if it comes down heads, his stake is return and he wins an amount equal to it. He decides to follow a simple strategy: if he has less than $£5$, he stakes everything, if he has more, he stakes just enough to get $£10$ if he wins.

I let $X_n$ be the gambler's capital after the $n$th coin toss (in $£$).
I've already worked out some results, played along with the situation, the state space is limited $\lbrace0,2,4,6,8,10\rbrace$ and we can write a transition matrix

$\begin{pmatrix}
1&0&0&0&0&0\\
1/2&0&1/2&0&0&0\\
1/2&0&0&0&1/2&0\\
0&1/2&0&0&0&1/2\\
0&0&0&1/2&0&1/2\\
0&0&0&0&0&1\\
\end{pmatrix}$

I'm trying to find out about the distribution of the whole sequence $X_0,X_1,…$ conditional on the event that he reaches $£10$ at some point.

I've tried an easy example such as $\mathbb{P}(X_1=10 | X_n=10$ for some $n\geq1,[X_0=8])$ but I don't get anywhere… Do you have an idea?

Any help would be amazing, thank you!

Best Answer

The result is a Markov chain on the state space $\{2,4,6,8,10\}$ with transitions:

  • $8\to6$ and $8\to10$ with probabilities $\frac38$ and $\frac58$ respectively
  • $6\to2$ and $6\to10$ with probabilities $\frac16$ and $\frac56$ respectively
  • $2\to4$ and $4\to8$ both with probability $1$.
Related Question