Solved – How to check if randomization was proper

clinical-trialshypothesis testingrandom allocation

Randomized controlled trial (RCT), where participants are randomly allocated to 2 or more groups and given different treatments or interventions, to evaluate which one is better, are commonly performed.

For these RCTs, it is now recommended that hypothesis testing for differences in baseline parameters need not be done, since any differences will be due to chance alone.

  1. However, there could be various reasons due to which randomization process may fail or may not have been proper. What steps should be taken to see that this is not the case before analyzing data from any RCT.

  2. At the same time, it is recommended that baseline parameters should be checked to see "if effect sizes are large and meaningful" (see here). However, this is often not done in reports of RCTs. Should this not be emphasized as an essential step?

To clarify, we are talking about studies done by others which we have to review. If it is not possible to test adequacy of randomization after study is done, should we not perform hypothesis testing on key baseline parameters to find if there are any important differences between the groups?

Thanks for your insight.

Best Answer

The best way to generate a "reproducible randomisation" is to use a scripted randomisation algorithm in a statistical programming language and make sure you "set the seed" for the randomisation. For example, suppose you want a reproducible randomisation in R where you select $t=40$ random treatment objects from a larger group of $n=100$ objects. This could be implemented with code like this.

#Generate randomisation
#Seed was determined from a call to runif(1)*.Machine$integer.max
set.seed(1825154847)
t    <- 40
n    <- 100
RAND <- sort(sample.int(n, size = t, replace = FALSE))

#Show the values of the treatment group
RAND

[1]   6   9  16  18  20  22  23  28  30  31  32  35  39  40  42  43  44  45  47  50
     53  64  67  68  71  72  73  75  79  82  83  84  85  86  90  93  94  96  99 100

As to how you know that this is a "proper randomisation", this hinges on two things: (1) your trust in the underlying pseudo-random number generator in the software you are using; and (2) your trust that the seed was determined independently of the data. With regard to the first issue, the sampling methods in statistical languages like R are built on pseudo-random number generators that have to pass suites of "randomisation tests". These tests measure the outputs of the PRNG against various statistical tests to ensure that there is no significant evidence of non-random patterns. This is usually sufficient to give the user confidence that there are no systematic patterns in the PRNG that will bias the RCT. With regard to the second issue, this is largely a matter of trusting the person who wrote the test, though if this is an issue then it is possible to impose further protocols for choosing the seed (e.g., using a true random number generator with independent witnesses).

With regard to your second point I am not sure what you mean, but I certainly agree with the view that post hoc tests of randomisatoin are discouraged and should not be used. When engaging in statistical hypothesis tests for "effects" of a treatment, the usual caveats on results should be borne in mind --- particularly the difference between "statistically significance" (i.e., significant evidence of a non-zero effect size), versus "practical significance" (i.e., substantial magnitude of the effect size). The randomisation in the RCT is a protocol to remove causal pathways from would-be confounding variables to the treatments, to assist in making causal inferences from the statistical inferences. It does not change the normal caution required for the interpretion of the statistical aspects of the inferences.

Finally, it is important to bear in mind that if one wishes to randomise treatments, but still "balance" some covariates, this can be done using block randomisation. This is the proper randomisation method to use if you want to balance some covariate while still desirable randomisation properties --- it is far superior to doing a simple-random-sample and then performing post hoc checks for balance. For example, if you have an overall group that is equally composed of males and females, and you want a treatment and control group to also have equal numbers of males and females, you can do this by forming "blocks" of male-female pairs and then randomly allocating to the groups from these blocks. (A full description of the process is beyond the scope of this answer, but look it up if you are interested.)

Related Question