From One Thousand Exercises in Probability I found the exercise:
Galton's paradox. You flip three fair coins. At least two are alike, and it is an even chance that the third is a head or a tail. Therefore $\mathbb{P}(\text{All Alike}) = \frac{1}{2}$. Do you agree?
Not only do I not agree, but I am having trouble identifying what the question was getting at to be a paradox.
Mapping Heads:=1 and Tail:=0, we can tabulate the possibilities for the three coins.
| Coin 1 | Coin 2 | Coin 3 |
|---|---|---|
| 0 | 0 | 0 |
| 0 | 0 | 1 |
| 0 | 1 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 0 |
| 1 | 0 | 1 |
| 1 | 1 | 0 |
| 1 | 1 | 1 |
There are eight possibilities of which only two of them match the criteria of the problem. Thus the probability $\mathbb{P}(\text{All Alike}) = \frac{2}{8} = \frac{1}{4}$.
My understanding is that a paradox should have two or more reasonable-sounding interpretations that should at least seem to contradict each other. My guess is that I missing something having to do with "At least two are alike, and it is an even chance that the third is a head or a tail.", but those are both true under the approach I took in getting $\mathbb{P}(\text{All Alike}) = \frac{1}{4}$.
Not that it is a requirement of a paradox, but often they give me a spinning dancer experience where I see it both ways for a while. Something about my understanding of this problem feels off.
What is the other side of this paradox that I am not seeing?
Best Answer
The "paradox" here arises from sneaking in an implicit insinuation of independence that does not actually hold, which allows the argument to lead you to a wrong answer with a series of plausible-sounding steps. As you can see from your enumeration of the possible outcomes, every possible outcome is one where "at least two are alike" so conditioning on this event has no effect on the probabilities (it is equivalent to conditioning on the sample space). However, by then looking at an event pertaining to "the third" coin we are implicitly referencing an event that depends on which two of the previous values are alike. Consequently, it turns out that the outcome of "the third" coin is not independent of the shared outcome of the other two coins. While it is (marginally) true that "it is an even chance that the third is a head or a tail", this no longer holds once we condition on the outcome of the other two coins.
To see this, let's proceed through the enumeration and look at the outcome of "the third" coin compared to the other two. For each row $k$ let $R_k$ denote the coin that is designated as "the third" coin in the outcome (which can take on multiple values in some cases), let $S_k$ denote the outcome of "the two" coins, and let $T_k$ denote the outcome of "the third" coin. Going through the enumerated rows we get:
$$\begin{matrix} R_1 = 1,2,3 & & & S_1 = 0 & & & T_1 = 0 & & & S_1 = T_1, \\ R_2 = 3 \quad \quad & & & S_2 = 0 & & & T_2 = 1 & & & S_2 \neq T_2, \\ R_3 = 2 \quad \quad & & & S_3 = 0 & & & T_3 = 1 & & & S_3 \neq T_3, \\ R_4 = 1 \quad \quad & & & S_4 = 1 & & & T_4 = 0 & & & S_4 \neq T_4, \\ R_5 = 1 \quad \quad & & & S_5 = 0 & & & T_5 = 1 & & & S_5 \neq T_5, \\ R_6 = 2 \quad \quad & & & S_6 = 1 & & & T_6 = 0 & & & S_6 \neq T_6, \\ R_7 = 3 \quad \quad & & & S_7 = 1 & & & T_7 = 0 & & & S_7 \neq T_7, \\ R_8 = 1,2,3 & & & S_8 = 1 & & & T_8 = 1 & & & S_8 = T_8. \\ \end{matrix}$$
As you can see, if we take the rows to be equiprobable (which occurs with flipping of independent fair coins) then it is true that $\mathbb{P}(T=0) = \mathbb{P}(T=1) = \tfrac{1}{2}$ (i.e., the third coin is equally likely to be a head or tail). However, the random variables $S$ and $T$ are not independent. In fact, we have:
$$\begin{matrix} \mathbb{P}(T=0|S=0) = \tfrac{1}{4} & & & \mathbb{P}(T=0|S=1) = \tfrac{3}{4}, \\[6pt] \mathbb{P}(T=1|S=0) = \tfrac{3}{4} & & & \mathbb{P}(T=1|S=1) = \tfrac{1}{4}. \\[6pt] \end{matrix}$$
Consequently, the probability that all the coins have the same outcome is:
$$\begin{align} \mathbb{P}(S=T) &= \mathbb{P}(S=T|S=0) \cdot \mathbb{P}(S=0) + \mathbb{P}(S=T|S=1) \cdot \mathbb{P}(S=1) \\[6pt] &= \mathbb{P}(T=0|S=0) \cdot \mathbb{P}(S=0) + \mathbb{P}(T=1|S=1) \cdot \mathbb{P}(S=1) \\[6pt] &= \tfrac{1}{4} \cdot \tfrac{1}{2} + \tfrac{1}{4} \cdot \tfrac{1}{2} \\[6pt] &= \tfrac{1}{4}. \\[6pt] \end{align}$$
We can see from this working that the marginal probability $\mathbb{P}(T=0) = \mathbb{P}(T=1) = \tfrac{1}{2}$ does not mean that $\mathbb{P}(S=T) = \tfrac{1}{2}$ (which is what the paradox is trying to convince you of). The reason for this is that $S$ and $T$ are not independent.
As has been pointed out in the comments on the question, there is some ambiguity in the identity of "the third" coin in this problem. However, that is not really affecting anything --- we can see that the probabilities in question are invariant to the way we select "the third" coin in the two ambiguous cases. The real reason for the "paradox" is the fact that the outcome of "the third" coin is not independent of the outcome of "the two" coins with shared outcomes (irrespective of how the third coin is chosen in cases of ambiguity). The argument in the paradox works by appealing to the intuitive plausibility of independence, when in fact it does not hold.