As you say, the chance that any number is on at least one die is $\frac {11}{36}$. At each point you have a favorite number, the one that you will keep in preference to any other. You will keep that one with probablity $\frac {11}{36}$. You will keep your second favorite with probability $\frac 9{36}$ because it loses two cases to the faovrite and so on down the scale.
Clearly you should always keep a $6$ if available. It give the most points and least restriction on what happens after.
As Empty2 suggested, think about the $9$th round. If you keep a $5$ your expected score in the last two rounds is $5+\frac {11}{36}\cdot6=\frac {246}{36}$ because you have to roll a $6$ in the tenth round to score. If you keep a $4$ your expected score is $4+\frac 9{36}\cdot 5 +\frac {11}6=\frac {255}{36}$. This is slightly better. Keeping a $3$ gets you $3+\frac 7{36}\cdot 4 +\frac 9{36}\cdot 5 +\frac {11}6=\frac {247}{36}$ Keeping a $2$ gets you $2+\frac 5{36}\cdot3+\frac 7{36}\cdot 4 +\frac 9{36}\cdot 5 +\frac {11}6=\frac {226}{36}$. Keeping a $1$ gets you $1+\frac 3{36}\cdot 2+\frac 5{36}\cdot3+\frac 7{36}\cdot 4 +\frac 9{36}\cdot 5 +\frac {11}6=\frac {196}{36}$
This shows you should keep the numbers on round $9$ in preference order $6,4,3,5,2,1$ We can make a spreadsheet showing the expected score from this round and later if you keep each number. For each round your favorite number to keep is a $6$, then is the one that gives the next highest score going forward. Through round $6$ the rule is simple-keep a $6$ and otherwise keep the smallest number you can. In rounds $7,8,9$ the order is confused by the short game and in round $10$ you should keep the largest number you can.
If there are more sides on the dice the perturbation at the end may get longer, but I suspect the early idea of take the smallest number available will still hold. If you roll more dice the chance of getting at least one $6$ gets higher and it may well be better to keep higher numbers. With lots of dice you will almost always get a $6$ so if you don't you should clearly keep the highest number available, which will very likely be a $5$.
Rounds are across the top, number kept down the left column. Entries are the expected score from this round forward if you keep the number to the left
I think there is not a general solution. Here is my solution on 4 players, rolling 4-sided dice case. Calculation is pretty complicated. I don't know if there is any better method.
First of all, we need to figure out results of each round of rolling. 4 players roll 4 dice, then we can get:
- 0 winner: $p_0=\frac{10}{64}$
- AAAA: $p_01=4*\frac{1}{4^4}=\frac{1}{64}$
- AABB: $p_02={4\choose2}\frac{\frac{4*3}{2}}{4^4}=\frac{9}{64}$
- 1 winner: $p_1=\frac{12}{64}$
- AAAB: $p={4\choose1}\frac{4*3}{4^4}=\frac{12}{64}$
- 2 winners: $p_2=\frac{36}{64}$
- AABC: $p={4\choose2}\frac{4*3*2}{4^4}=\frac{36}{64}$
- 3 winners: $p_3=0$
- 4 winners: $p_4=\frac{6}{64}$
- ABCD: $p=\frac{4*3*2*1}{4^4}=\frac{6}{64}$
$E(n)$ is the expected number of rolls to finish a game where $n$ players have not yet won at least once. So we want to know $E(4)$.
$$E(4)=1+p_0*E(4)+p_1*E(3)+p_2*E(2)$$
$$E(3)=1+p_0*E(3)+\frac{1}{4}p_1*E(3)+\frac{3}{4}p_1*E(2)+\frac{1}{2}p_2*E(2)+\frac{1}{2}p_2*E(1)$$
$$E(2)=1+p_0*E(2)+\frac{2}{4}p_1*E(2)+\frac{2}{4}p_1*E(1)+\frac{1}{6}p_2*E(2)+\frac{4}{6}p_2*E(1)$$
$$E(1)=1+p_0*E(1)+\frac{3}{4}p_1*E(1)+\frac{3}{6}p_2*E(1)$$
And I got $E(4)=6.401612$ which is not close to the simulation result. If you find out what's wrong with it please comment. Thanks a lot!
Best Answer
For standard six-sided dice, $v_{199}-v_{198}\approx6+9\cdot10^{-21}$. Here's the code I used to compute this value. The numbers are represented exactly as rationals, so rounding errors are excluded.
The basic idea is to keep track of two values for each $n$: the value $v_n$ of the game with $n$ dice and the value $w_n$ of the game when preceded by a free roll without $6$s and without the requirement to fix at least one die. For $n\ge6$, we have $v_n\gt5n$, so the free roll becomes worthless and $w_n=v_n$. Thus we only have to calculate $w_n$ and $v_n$ separately up to $n=5$, and as long as the optimal strategy fixes all $6$s, for $n\gt6$ we can calculate $w_n=v_n$ as
\begin{align} w_n &=6^{-n}\left(5^n(w_{n-1}+5)-\sum_{d=1}^4d^n+\sum_{k=0}^{n-1}\binom nk5^k\left(w_k+6(n-k)\right)\right)\\ &=C_n+6^{-n}\left(\sum_{k=0}^n\binom nk5^kw_k-5^n(w_n-w_{n-1})\right)\;, \end{align}
where in the first equation the first term corresponds to the case without $6$s, the second term corrects for the fact that the highest die in that case need not be a $5$, and the sum in the third term is over the number $k$ of non-$6$s; and where
$$C_n=n+6^{-n}\left(5\cdot5^n-\sum_{d=1}^4d^n\right)\;,$$
as defined in Milo Brandt's answer, is the expected value of the dice to be fixed either as $6$s or as the highest die. (This is essentially the modification of Milo's approach to account for the exceptions at small $n$, as suggested in his answer.)
Since the calculation yields $v_{199}-v_{198}\gt6$, the optimal strategy for $200$ dice does not fix the second of exactly two $6$s. Beyond that, the calculation is no longer valid, since it assumes that $6$s are always fixed.
I added this question here as an answer to Examples of apparent patterns that eventually fail.
I'm leaving the original answer up in case the process that led to the solution is of any interest.
Here are some results vor $v_n-v_{n-1}$ for other dice. They don't settle the concrete question for standard six-sided dice, but they indicate why it's so difficult to settle, and they disprove the intuition discussed in the question that the highest number should never be re-rolled.
For three-sided dice with numbers $0,1,1$:
and so on, approaching $1$ from above until at least $n=50$. So in this case $1$s should always be rerolled, except if all dice are $1$s. But yet more interesting, for three-sided dice with numbers $1,2,3$:
(Scroll down to see the results up to $n=50$.) The values are below $3$ for $1\le n\le27$, then above $3$ for $28\le n\le34$, then below $3$ for $35\le n\le40$ and then above $3$ until at least $n=72$, approaching it from above after a local maximum at $n=44$.
These results were obtained with the same code that reproduces Joachim's and Byron's results for standard dice up to $n=7$, so a bug is unlikely.
The results show that general strategy stealing arguments (as suggested by mercio in a comment; I'd also tried to find one) will not work and any proof will have to be based on the specific numbers on the six-sided dice. (A six-sided die with one low and five high numbers behaves much like the three-sided die with one low and two high numbers.)
P.S.: Here are the results for two-sided dice (a.k.a. coins) with numbers $0,1$:
and so on, approaching $1$ from above roughly exponentially until at least $n=60$.
I'm now running a longer calculation with standard four-sided dice to see whether they eventually follow the pattern of the three-sided dice. So far, up to $n=50$, the differences seem to be approaching $4$ from below roughly exponentially. But note that for the three-sided dice with numbers $1,2,3$, the differences appear to approach $3$ from below roughly exponentially all the way right up to $n=27$, where the difference from $3$ is only $2\cdot10^{-9}$, and then they decide to exceed $3$ after all; so all bets are off.
$Update$: For standard four-sided dice, $v_{70}-v_{69}\approx4-1.5\cdot10^{-13}$ and $v_{71}-v_{70}\approx4+1.7\cdot10^{-13}$. This suggests that we may eventually have $v_n-v_{n-1}\gt6$ for standard six-sided dice after all; and since the switch occurs at $n=28$ for three-sided dice and at $n=70$ for four-sided dice, it may occur at quite high $n$ well over $100$ for six-sided dice. Perhaps the effort we've been putting into proving $v_n-v_{n-1}\lt6$ should better be directed at making larger values of $n$ numerically accessible.