I was wondering a very simple question as I was reading a book about predictions.
Let's say you have a coin and you flip it twice, and both times it comes up heads. How could you calculate the odds that the coin is fair? Is this even possible?
If we look at a more extreme example, let's say we toss a coin a million times, and we get 1 million heads. What are the odds the coin is fair? I would be extremely confident at that point that the coin is NOT fair, 99%+ confident (a made up number). However, even here I would be unsure how to assign a mathematical answer.
I know the odds of getting 1 million heads in a row is $(.5)^(1million)$ which is basically so intuitively maybe 1- $(.5)^(1million)$ is the odds it's an unfair coin.But that seems wrong when we extend it to a 2 flip example, since the odds of a getting two heads in a row is 25% and it seems absurd after two consecutive heads to declare a coin 75% likely to be biased.
Now I have seen the Bayes' way of doing it, where we start with a prior probability. (if there is anything wrong with what I post below please let me know)
If we assume there's a 50-50 chance of a coin being fair or biased such that when it is tossed heads will come up everytime.
Now it becomes a simple matter. $1/2$ of the time we will get 2 heads from the biased coin, and $1/2$ * $1/4$ = $1/8$ times we get the fair coin giving us two heads.
Now we can sum these two up and get $5/8$ chance of the coin coming up heads twice in a row. And so the odds of the coin being biased is $(4/8) / (5/8)$ = 80%
However, that number depends on our initial estimate of the coin being biased. Is it possible to have some estimate of the biased coin without an initial estimate? in the case of a million consecutive flips, I feel like you can say with almost 99.999% (arbitrary number but probably close to the "true" answer whatever that is) certainty the coin is not fair without any need for a prior estimate.
Best Answer
As Arnaud alluded to, you cannot go anywhere unless you have other coins of some probability of being selected with some probability of being heads. On the other hand, what you can do is test the null hypothesis that the coin was fair, after you observed flipping $2$ heads.
You can implement a binomial test. Assume the coin is fair.
Let $$H_0 : p=0.5$$
$$H_a : p\neq 0.5$$
Then what is the probability of getting $2$ heads or $2$ tails?
Let $X$ be the number of heads we observe. Then
$$P(X=0)+P(X=2)=2\cdot P(X=2)=2\cdot{0.5^2}=0.5$$
Thus we fail to reject the null hypothesis at a meaningful significance level that the coin was fair.
A more interesting example would be increasing the number of trials to $n=10$
Here, if you flipped $10$ heads then you'd reject the null hypothesis at $\alpha=0.01$ that the coin was fair since $$P(X=0)+P(X=10)=0.5^9\approx0.00195\lt0.01$$