First, some context. I have a turn-based strategy game where pieces on a board can attack other pieces. Every attack has a percentage chance of success or failure. This percentage can vary. Using normal RNG techniques, this can result in unbalanced "luck" where one player succeeds more often and under less likely conditions than the other player.
There are multiple ways to improve the balance of "luck" in this game, but the one I'm looking at right now is to boost or reduce the percentage chance of the next attempt based on prior attempts. This is one way to allow a player to more reliably expect the next attempt to succeed after a string of failures.
In my research, I looked up coin flips, e.g. "What are the odds of getting heads at least once in 3 tries". I found a formula for calculating that, which looks like this:
$odds = 1 – 0.5^3 = 0.875$
So, I'm not sure how to apply that to my context. But, perhaps this example will help you understand what I'm after. In short, I know that the chances of getting heads on the 6th try is 50% even after flipping a coin 5 times and getting T, T, H, T, and H. But I want to slightly boost those chances based on the fact that I already flipped slightly more tails than heads previously (or reduce those chances if the reverse is true). Also, in my context, the chances aren't always 50/50 like coins. So I need some sort of formula where I can input prior attempt percentages to arrive at an adjusted percentage for the next attempt.
So here's some example input that should be a good test for the formula. Even if I have 2 failed attacks and 1 successful attack, the odds of success will still be REDUCED for the next attack because the success was less likely than the 2 failures combined. Here they are in order:
- 1st attempt: 10% chance of success, but it succeeded
- 2nd attempt: 20% chance of success, but it failed
- 3rd attempt: 25% chance of success, but it failed
- 4th attempt: 50% chance of success on an individual basis, but what are the odds on an aggregate basis?
Final Comments:
- I don't expect the order of prior attacks to matter when computing the aggregate odds for the next. But I am interested in seeing a counter argument and formula.
- I expect we only need to input "individual" percentages (as opposed to "aggregate" percentages) for prior attacks when computing the aggregate percentage for the next. But, again, I am interested in seeing a counter argument and formula.
Best Answer
I think it might be possible to come up with multiple different "correction" strategies since the requirements are too general for now.
Setup
Assume $t\in\mathbb{Z}$ denotes the time step. You start at $t=1$. The "base" probability of success at the time step $t$ is $p_t$ without any correction and the corrected version is $\hat p_t$. Let $X_t \in \{0, 1\}$ denote the outcome at time $t$, where $1$ is a success.
Probability of success
If you do not do any adjustment, the probability of succeeding at time step $t$ is $p_t$. You can write the probability of the outcome (whichever outcome you get) at time step $t$ to be $X_tp_t + (1-X_t)(1-p_t)$ which you can check gives you the correct answer irrespective of what $X_t$ is.
At time step $t$, the probability of exactly getting the sequence $X_1, X_2, ... , X_t$ (eg: $1, 0, 0, ... 1$) is (assuming independent flips) $\prod\limits_{i=1}^{t}\left(X_ip_i+(1-X_i)(1-p_i)\right)$. But since this is specific to a given order of outcomes, the probability monotonically decreases as $t$ increases (i.e. the number of possible sequences increase) as long as $p < 1$. At this stage, it is unclear how you would construct a probability question to ask some version of "how lucky the person was". You can try asking the probability of being as "lucky" as the person was or greater by asking the probability of getting as many successes the person got or more if you did not have varying $p$. This formulation for a time-dependent probability without a simple time-equation will require you to iterate over many different permutations (depending on how you define lucky) and might not be the best motivated question in your case anyway.
Considerations
Even in the above strategy, once you do come up with a "correction", it is difficult to decide whether you should use the uncorrected $p_t$ or the corrected version $\hat p_t$ to calculate future corrections. The former seems to be slightly better motivated since your aim is to find out how "lucky" the player has been in comparison to the base probabilities.
Easier Strategy
Since you only need a correction which roughly translates to the inverse of how lucky the person was, here is one way to do it. You find the series $E[X_1], E[X_2], ... , E[X_t]$ and then you "subtract" the sequence you obtained from it to obtain the sequence of deviations $p_1 - X_1, p_2 - X_2, ..., p_t - X_t$ (since $E[x_i]=p_i$, you can replace the former with the latter). This represents how much did the player deviate from the "expected" mean, negative deviation meaning the player was overly lucky. You can sum all the entries, and the sign of the sum tells you if the player was overly unsuccessful or overly successful. You can create a correction by adding a small portion of this sum to the next probability $p_{t+1}$. Ideally, you should also divide by the standard deviations to get a "normalized" version of the deviation from the mean of the individual sum elements. Here's one possible formula for the correction factor $c$ and some arbitrary small constant $a$.
$$c_{t+1} = \sum\limits_{i=1}^{t}\frac{p_i - X_i}{\sqrt{p_i(1-p_i)}}$$ $$\hat p_{t+1} = p_{t+1} + ac_{t+1}$$
(taking measures to keep $\hat p_{t+1}$bounded between $0$ and $1$) Selection of $a$ is in essence arbitrary, but here is a suggestion of how to come up with a value. You can simply set $a=\sqrt{p_{t+1}(1-p_{t+1})}$ which is the standard deviation of the $t+1$th step. You can keep a factor of $0<x \leq 1$ depending on how strict you want the correction to be.
Does this fulfill your requirements?
You can check out that this works for your given example. Note that the order of the successes do matter in this strategy. I am not sure what you mean by individual vs aggregate in the second point.
There are many specific properties of such a formula, and whether they suit you or some other suggestion might suit you better, depends on the details of the situation. From the broad phrasing of the question, I am assuming that an intuitively appeasing correction might work without the need of some specific statistical properties. You can try titrating the $a$ or $x$ to your needs, or try modifying the formula by taking the squared deviation from the mean divided by the variance, or forget the denominator entirely. All of these would change the statistical properties and may be better or worse (or equivalent) for your case.