Here's a dynamic dice game. We can roll dice as many times as we want unless a 6 appears. After each roll, if 1 appears, we win \$1, if 2 appears we win \$2 etc. If 6 appears, we lose all money and game ends. We can decide if we want to continue the game after witnessing anything between 1 and 5. How much would you be willing to pay to play the game?
My approach:
I first computed the expected payoff of the game. Sa $x$ is the expected payoff. Then we have
$$x = \frac{1}{6}(1+x)+\frac{1}{6}(2+x)+\frac{1}{6}(3+x)+\frac{1}{6}(4+x)+\frac{1}{6}(5+x) $$
This gives me $x=15$.
What next? How to find the optimal price to play the game before it begins? Thanks.
Best Answer
A strategy is simply given by a set $S\subset\Bbb N$ and amount to "If we have collected $x$ then stop of $x\in S$ and continue if $x\notin S$". We assume there is some optimal strategy $S_0$.
Let $E(x)$ be the expected final payout if we have already collected $x>0$ and make optimal decisions from now on. Clearly, we have $$\tag1 E(x)=\begin{cases}x&x\in S_0\\ \frac{E(x+1)+E(x+2)+E(x+3)+E(x+4)+E(x+5)}6&x\notin S_0\end{cases}$$ With strategy $S=\Bbb N$, we will of course stop immediately and have payout $x$. Therefore, $$\tag2 E(x)\ge x.$$ With $(1)$, this gives us the lower estimate $$ E(x)\ge \frac{E(x+1)+\cdots+E(x+5)}6\ge \frac{(x+1)+\cdots+(x+5)}6=\frac {5x+15}6,$$ which is certainly $>x$ if $x<15$. In other words, $\{1,\ldots,14\}\cap S_0=\emptyset$.
If $x>k$, let $S=S_0+k=\{\,s+k\mid s\in S_0\,\}$. Then strategy $S$ shows $$E(x)\le E(x-k)+k$$ (namely, we make our decision as if there were $k$ less, hence gain $k$ more than "expected" if we win something, or end up with the same $0$ if we lose)
Combining with $(1)$, we find for $x\in S_0$ that $$ E(x)\le\frac{5E(x)+15}6$$ and hence $$ E(x)\le 15.$$ As also $E(x)\ge x$, we conclude that $x\in S_0$ if $x>15$. So $$ E(x)=x\quad\text{if }x\ge 15.$$ This makes $$ E(14)=\frac{E(15)+\cdots+E(19)}6=14\tfrac16$$ $$ E(13)=\frac{E(14)+\cdots+E(18)}6=13\tfrac{13}{36}$$ $$ E(12)=\frac{E(13)+\cdots+E(18)}6=12\tfrac{127}{216}$$ and so on until $$ E(0)=\frac{E(1)+\cdots+E(5)}6=13\tfrac{13}{36}=6\tfrac{ 72285265495}{470184984576}\approx 6.1537.$$
Hence the fair price for the game is $\approx 6.15$ and the best strategy is to keep playing until one reaches $15$ or more.