I am asking this question because of a recent question on Skeptics.SE and a 2015 study by Duke University and the Federal Reserve Bank of Boston. This question is based on the study.
In a Boston study on the racial wealth gap, researchers wrote (emphasis added):
Whereas white households have a median wealth of $247,500, Dominicans
and U.S. blacks have a median wealth of close to zero (see Table 9).
Somebody asked if there were faults in the methodology of the study, considering only the statistics for whites and "U.S. blacks". The study surveyed 78 whites and 71 U.S. blacks. Assuming that the population size is 4.6 million residents, is the sample size reasonable?
Assume that I know little to nothing about statistics.
Best Answer
I would say that the sample sizes are too small. In each group (Black and White), there will be considerable diversity. There are subpopulations. I think that having fewer than 80 subjects in each group means that the samples will miss some of those subpopulations. Even if they get statistically significant results, I am skeptical that enough subpopulations have been sampled.
However, looking at the graphs, this appears to be a case where we don't need statistical inference tools to make a conclusion.
Edit:
A typical gripe about small sample sizes is that you don't give your test a fighting chance to reject the null hypothesis. If you flipped a coin four times and got HHTH, would you be so confident in concluding that the coin is biased toward heads? I would not be. If you flipped that same coin 10,000 times and got 7,500 heads and 2,500 tails, you'd be much more confident in saying that the coin is biased. Your first test with four observations was unable to reject the false null hypothesis of equal probability of heads and tails, but this is beause you didn't have enough observations (coin flips) to give the test adequate power.
What this means is that, if the study concluded from ~150 subjects that there is no evidence to suggest a difference between Black and White people, a legitimate criticism could be that only 150 people were sampled, giving the test inadequate power to reject a null hypothesis that both groups have the same median. The test was inconclusive, but only because of inadequate sample sizes, not because of how the populations are distributed. However, the study was able to reject that null hypothesis. The study had a large enough sample size to give it adequate power to reject.