Random assignment is valuable because it ensures independence of treatment from potential outcomes. That is how it leads to unbiased estimates of the average treatment effect. But other assignment schemes can also systematically ensure independence of treatment from potential outcomes. So why do we need random assignment? Put another way, what is the advantage of random assignment over nonrandom assignment schemes that also lead to unbiased inference?
Let $\mathbf{Z}$ be a vector of treatment assignments in which each element is 0 (unit not assigned to treatment) or 1 (unit assigned to treatment). In a JASA article, Angrist, Imbens, and Rubin (1996, 446-47) say that treatment assignment $Z_i$ is random if $\Pr(\mathbf{Z} = \mathbf{c}) = \Pr(\mathbf{Z} = \mathbf{c'})$ for all $\mathbf{c}$ and $\mathbf{c'}$ such that $\iota^T\mathbf{c} = \iota^T\mathbf{c'}$, where $\iota$ is a column vector with all elements equal to 1.
In words, the claim is that assignment $Z_i$ is random if any vector of assignments that includes $m$ assignments to treatment is as likely as any other vector that includes $m$ assignments to treatment.
But, to ensure independence of potential outcomes from treatment assignment, it suffices to ensure that each unit in the study has equal probability of assignment to treatment. And that can easily occur even if most treatment assignment vectors have zero probability of being selected. That is, it can occur even under nonrandom assignment.
Here is an example. We want to run an experiment with four units in which exactly two are treated. There are six possible assignment vectors:
- 1100
- 1010
- 1001
- 0110
- 0101
- 0011
where the first digit in each number indicates whether the first unit was treated, the second digit indicates whether the second unit was treated, and so on.
Suppose that we run an experiment in which we exclude the possibility of assignment vectors 3 and 4, but in which each of the other vectors has equal (25%) chance of being chosen. This scheme is not random assignment in the AIR sense. But in expectation, it leads to an unbiased estimate of the average treatment effect. And that is no accident. Any assignment scheme that gives subjects equal probability of assignment to treatment will permit unbiased estimation of the ATE.
So: why do we need random assignment in the AIR sense? My argument is rooted in randomization inference; if one thinks instead in terms of model-based inference, does the AIR definition seem more defensible?
Best Answer
This follows up on gung's comment. Overall average treatment effect is not the point.
Suppose you have $1000$ new diabetes cases where the subject is between the ages of $5$ and $15$, and $1000$ new diabetes patients over $30$. You want to assign half to treatment. Why not flip a coin, and on heads, treat all of the young patients, and on tails, treat all of the older patients? Each would have a $50\%$ chance to be selected fro treatment, so this would not bias the average result of the treatment, but it would throw away a lot of information. It would not be a surprise if juvenile diabetes or younger patients turned out to respond much better or worse than older patients with either type II or gestational diabetes. The observed treatment effect might be unbiased but, for example, it would have a much larger standard deviation than would occur through random assignment, and despite the large sample you would not be able to say much. If you use random assignment, then with high probability about $500$ cases in each age group would get the treatment, so you would be able to compare treatment with no treatment within each age group.
You may be able to do better than to use random assignment. If you notice a factor you think might affect the response to treatment, you might want to ensure that subjects with that attribute are split more evenly than would occur through random assignment. Random assignment lets you do reasonably well with all factors simultaneously, so that you can analyze many possible patterns afterwards.