This answer is longer than it needs to be because I wanted to make a general argument. This will work for any sided die. I could have generalized it slightly more in terms of how much you have to pay to roll again but I got lazy. Feel free to ask clarifying questions!
At every point in the game, your strategy should be irrespective of your previous rolls. To see why this is, think about it like this. Whatever you rolled previously has no effect on you right now. That is, no matter how much you have lost or won, if this current roll you are about to do is profitable, you should do it whether you have already won a million dollars or lost a trillion, because at this current moment in time, it is absolutely profitable. Your past earnings play no roll in the decision.
Our strategy will be to choose a cutoff number above which we will accept the winnings and stop. For this cutoff, we will develop a way to calculate the expected winnings and maximize them.
Let $n$ be the number of sides on the dice and $k$ be the cutoff. We will calculate the expected value of this strategy ($S_k$) as:
$$
E[S_k] = p(x \geq k)\cdot E[ x\vert x \geq k] + (1-p(x\geq k))\cdot \left(E[S_k]-1\right)
$$
That is, we have agreed if our roll is greater than the cutoff we will stop and if it is less we will roll again. So if our roll is greater than $k$ (which will happen with $p(x \geq k)$), then we calculate the expected value of the values of $x \geq k$ and multiply this by the probability. If however, our roll is less than the cutoff, then we are back where we started except we have lost a dollar so the expected value is $E[S_k] - 1$.
Rearranging the equation gives us:
$$
\begin{align}
E[S_k] &= p(x \geq k)\cdot E[ x\vert x \geq k] + (1-p(x\geq k))\cdot \left(E[S_k]-1\right) \\
E[S_k]\cdot p(x\geq k) &= p(x \geq k)\cdot E[ x\vert x \geq k] - (1-p(x\geq k)) \\
E[S_k] &= \frac{p(x \geq k)\cdot (1 + E[ x \vert x \geq k]) - 1}{p(x\geq k)} \\
E[S_k] &= 1 + E[ x \vert x \geq k] - \frac{1}{p(x\geq k)} \\
\end{align}
$$
Let us calculate $p(x \geq k)$ and $E[x \vert \geq k]$. There are $n-k+1$ rolls greater than or equal to $k$ and since there are $n$ possible rolls, we get:
$$
p(x \geq k) = \frac{n + 1 - k}{n}
$$
As for $E[x \vert x \geq k]$, the numbers $k, k+1, \ldots n$ are all greater than or equal to $k$ and occur with equal probability. So the expected value of these is just there average i.e.:
$$
E[x \vert x \geq k] = \frac{n + k}{2}
$$
Combining these, we finally get:
$$
E[S_k] = 1 + \frac{n + k}{2} - \frac{n}{n + 1 - k} \\
$$
We want to maximize this function. We can do this easily by computing when the derivative is $0$:
$$
\frac{\partial}{\partial k} E[S_k] = \frac{1}{2} - \frac{n}{(n + 1 - k)^2} \\
$$
and so, we compute when this is $0$:
$$
\begin{align}
(n + 1 - k)^2 - 2n &= 0\\
k^2 + k(-2 n - 2) + n^2 + 1 &= 0 \\
k &= \frac{2n + 2 \pm \sqrt{(2n+2)^2 - 4(n^2 + 1)}}{2} \\
k &= n + 1 \pm \sqrt{2n} \\
k &= n + 1 - \sqrt{2n}
\end{align}
$$
So, plugging in your values for $n = 6$, your final answer is $k = 3.53$ so as soon as you roll above $3$ in the game, you should stop :)
Addendum:
Just for completeness, the more general version, where you have to pay $p$ dollars to roll again is given by:
Optimum cutoff:
$$
k = n + 1 - \sqrt{2np}
$$
In case you wanted some evidence, here are the graphs of average money earned after stopping as soon as a roll of $x$ or higher is seen.
$n = 6$. Note how the optimal value is when $x = 3$ or $x = 4$ are the cutoffs:
$n = 50$. Note how max is at $51 - \sqrt{100} = 41$, exactly as the formula predicts:
I think there is not a general solution. Here is my solution on 4 players, rolling 4-sided dice case. Calculation is pretty complicated. I don't know if there is any better method.
First of all, we need to figure out results of each round of rolling. 4 players roll 4 dice, then we can get:
- 0 winner: $p_0=\frac{10}{64}$
- AAAA: $p_01=4*\frac{1}{4^4}=\frac{1}{64}$
- AABB: $p_02={4\choose2}\frac{\frac{4*3}{2}}{4^4}=\frac{9}{64}$
- 1 winner: $p_1=\frac{12}{64}$
- AAAB: $p={4\choose1}\frac{4*3}{4^4}=\frac{12}{64}$
- 2 winners: $p_2=\frac{36}{64}$
- AABC: $p={4\choose2}\frac{4*3*2}{4^4}=\frac{36}{64}$
- 3 winners: $p_3=0$
- 4 winners: $p_4=\frac{6}{64}$
- ABCD: $p=\frac{4*3*2*1}{4^4}=\frac{6}{64}$
$E(n)$ is the expected number of rolls to finish a game where $n$ players have not yet won at least once. So we want to know $E(4)$.
$$E(4)=1+p_0*E(4)+p_1*E(3)+p_2*E(2)$$
$$E(3)=1+p_0*E(3)+\frac{1}{4}p_1*E(3)+\frac{3}{4}p_1*E(2)+\frac{1}{2}p_2*E(2)+\frac{1}{2}p_2*E(1)$$
$$E(2)=1+p_0*E(2)+\frac{2}{4}p_1*E(2)+\frac{2}{4}p_1*E(1)+\frac{1}{6}p_2*E(2)+\frac{4}{6}p_2*E(1)$$
$$E(1)=1+p_0*E(1)+\frac{3}{4}p_1*E(1)+\frac{3}{6}p_2*E(1)$$
And I got $E(4)=6.401612$ which is not close to the simulation result. If you find out what's wrong with it please comment. Thanks a lot!
Best Answer
You can estimate it as follows: There are two possibilities - Either you roll a six, or you don't. If you don't, the expected gain is $\frac{1+2+3+4+5}{5}=3$ (Note that this is the expected gain GIVEN that you didn't roll a $6$). Now suppose that no matter what number you get from $1$ to $5$ you gain $\$3$. As stated in the solution you provided, you should keep rolling until you have $\ge 15$, so in this case you roll exactly $5$ times. If you are successful with all five rolls, you gain $\$15$, but if you rolled a $6$ in any one of these rolls you gain $\$0$.
So the estimate is $15*\left(\frac{5}{6}\right)^5=6.03$.