[Math] Why is observing 100 heads for a fair coin flips surprising

Question

Assume that we have a fair coin. We flip it 100 times. The outcome is all heads.

Why is it surprising?

Doesn't all outcomes have the same probability? Any particular outcome, including irregular ones, would have the same very small probability.

Why is it that observing an irregular outcome is less surprising to us than a regular one?

Answer 1

You are right. It is not that surprising. Each specific pattern has probability $\frac{1}{2^{100}}$. The probability to obtain $100$ heads is the same as obtaining any other particular outcome.

But usually we do not ask for a specific pattern but ask for the probability to obtain e.g. $k$ heads and this makes the difference.

• There is just $\color{blue}{\binom{100}{100}=1}$ pattern to obtain $100$ heads out of $2^{100} \sim 1.3\cdot 10^{30}$.

• But we have $\color{blue}{\binom{100}{50}\sim 1.0\cdot 10^{29}}$ patterns which contain $50$ heads and $50$ tails.