A two-sided coin has just been minted with two different sides (heads and tails). It has never been flipped before. Basic understanding of probability suggests that the probability of flipping heads is .5 and tails is .5. Unexpectedly, you flip the coin a very large number of times and it always lands on heads. Is the probability of flipping heads/tails still .5 each? Or has it changed in favor of tails because the probability should tend to .5 heads and .5 tails as you approach an infinite number of trials?
I understand that flipping coins is generally a stochastic process, but does that change at all if you see a large number of trials bias to one side?
Best Answer
If you don't know whether it is a fair coin to start with, then it isn't a dumb question at all. (EDIT) You ask if the coin will be biased towards Tails to account for the all of the heads. If the coin was fair, then the answer from tilper addresses this well, with the overall answer being "No". Without that assumption of fairness, the overall answer becomes "No, and in fact we should believe the coin is biased towards heads.".
One way to think about this is by thinking of the probability $p$ of the coin landing heads to be a random variable. We can assume that we know absolutely nothing about the coin to start with, and take the distribution for $p$ to be a uniform random variable over $[0,1]$. Then, after flipping the coin some number of times and collecting data, we change our distribution accordingly.
There is actually a distribution which does exactly this, called the Beta Distribution, which is a continuous distribution with probability density function $$f(x) = \frac{x^{\alpha-1}(1-x)^{\beta-1}}{B(\alpha,\beta)}$$ where $\alpha-1$ represents the number of heads we've recorded and $\beta-1$ the number of tails. The number $B(\alpha,\beta)$ is just a constant to normalize $f$ (however, it is actually equal to $\frac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha+\beta)}$).
The following graphic (from the wikipedia article) shows how $f$ changes with difference choices of $\alpha,\beta$:
As $\alpha \to \infty$ (i.e. you continue getting more heads) and $\beta$ stays constant (below I chose $\beta=5$), this will become extremely skewed in favor of $p$ being close to $1$.