Solved – Alternate weighting schemes for random effects meta-analysis: missing standard deviations

meta-analysismeta-regressionmissing datastata

I am working on a random effects meta-analysis covering a number of studies which do not report standard deviations; all studies do report sample size. I do not believe it is possible to approximate or impute the SD missing data. How should a meta-analysis which uses raw (unstandardized) mean differences as an effect size be weighted when standard deviations are not available for all studies? I can, of course still estimate tau-squared and would like to incorporate that measure of between-study variance in whatever weighting scheme I use to stay within the random-effects framework.

A little more information is included below:

  1. Why raw mean differences might still be useful:
    The data is reported in a an intrinsically meaningful scale: US dollars per unit. So, a meta-analysis of mean differences would be immediately interpretable.

  2. Why I cannot approximate or impute the SD data:
    The studies for which standard deviation data is missing do not include enough data to approximate a standard deviation (i.e. median and range are never reported in the literature). Imputing the missing data seems inadvisable as a large portion of the studies are missing the sd, and because the studies differ greatly in terms of geographical region covered and survey protocol.

  3. What is typically done with raw mean differences in meta-analysis:
    Study weights are based on the standard error of the mean difference (typically computed with sample-size term and the pooled variance). I don't have this. In a random-effects meta-analysis, study weights also include a term for between-study variance. I have this.

Can simple inverse-sample-size weighting be used in this context? How would I incorporate my estimate of tau-squared (or some other measure of between-studies dispersion) into the weighting?

Best Answer

If you meta-analyze a mean differences with weights of $n$ instead of by $1/\text{SE}^2$ (inverse variance) - assuming groups of equal size are being compared - this gets you an appropriate average effect estimate under the assumption that variability is the same across studies. I.e. the weights would be proportional to the ones you do would use, if the standard errors were all exactly $2\hat{\sigma}/\sqrt{n}$ for a standard deviation $\sigma$ that is assumed to be identical across trials. You will no longer get a meaningful overall standard error or confidence interval for your overall estimate though, because you are throwing away the information $\hat{\sigma}$ on the sampling variability.

Also note that if groups are not of equal size $n$ is not the correct weight, because the the standard error for the difference of two normal distributions is $\sqrt{\sigma^2_1/n_1 + \sigma^2_2/n_2}$ and this only simplifies to $2\sigma/\sqrt{n}$, if $n_1=n_2=n/2$ (plus $\sigma=\sigma_1=\sigma_2$).

You could of course impute the missing standard errors under the assumption that $\sigma$ is the same across the studies. Then studies without a reported standard error have the same underlying variability as the average of the studies, for which you know it and that's easy to do.

Another thought is that using untransformed US dollars or US dollars per unit might or might not be problematic. Sometimes it can be desirable to use e.g. a log-transformation to meta-analyze and then to back-transform afterwards.

Related Question