[Math] the expected number of flips that are needed

markov chainsstochastic-processes

Suppose we flip a fair coin repeatedly until we have flipped four consecutive heads. What is the expected number of flips that are needed?
The hint is given is as follows: Consider a Markov chain with state space {0,1,…4}.

Can anybody give an idea of where to start? Thanks! As I make progress I will add what my solution becomes. Otherwise any help would be appreciated.

Best Answer

Indeed let $X_n$ denote the number of consequtive heads that have flipped at time $n$. The possible states of $X_n$ are $\{0,1,2,3,4\}$ since if $4$ consecutive heads the process is considered to end. Given $X_n=j$ the next state can be either $0$ if we flip a tail or $j+1$ if we flip another consecutive head, thus $$X_{n+1}|X_n=j=\begin{cases} 0, &\text{ with probability } \frac{1}{2} \\ j+1, &\text{ with probability } \frac{1}{2} \end{cases}$$ since each event (head or tails) occurs with the same probability. Hence, the transition matrix of the markov chain is given below $$\begin{pmatrix}\frac{1}{2}&\frac{1}{2}&0&0&0\\\frac{1}{2}&0&\frac{1}{2}&0&0\\\frac{1}{2}&0&0&\frac{1}{2}&0\\\frac{1}{2}&0&0&0&\frac{1}{2}\\0&0&0&0&1\end{pmatrix}$$ where we have assumed that as long the game reaches state $4$ i.e. 4 consecutive heads the process seizes. The initial state is $X_0=0$. Let $h(j)$ denote the expected number of flips until you reach state $4$. Then you have the following system of equations $$\begin{align*}h(0)&=1+\dfrac{1}{2}h(0)+\dfrac{1}{2}h(1)\\h(1)&=1+\dfrac{1}{2}h(0)+\dfrac{1}{2}h(2)\\h(2)&=1+\dfrac{1}{2}h(0)+\dfrac{1}{2}h(3)\\h(3)&=1+\dfrac{1}{2}h(0)+\dfrac{1}{2}h(4)\\h(4)&=0\end{align*}$$ Now solve the above system recursively to obtain $h(0)$ which is the expected number of flips needed.


The solution is $h(0)=30$.

Related Question