Solved – Is stratified meta-analysis more or less objective than meta-regression

meta-analysismeta-regressionmethod-comparisonstratification

Reviewer asked me why I use meta-regression as a way how to deal with heterogeneity among effect sizes instead of conducting stratified meta-analysis.

I tried to google "stratified meta-analysis" and probably the most useful explanation was:

Stratification is an effective way to deal with inherent differences
among studies and to improve the quality and usefulness of the
conclusions. An added advantage to stratification is that insight can
be gained by investigating discrepancies among strata. There are many
ways to create coherent subgroups of studies. For example, studies can
be stratified according to their “quality,” assigned by certain
scoring systems. Commonly used systems award points on the basis of
how patients were selected and randomized, the type of blinding, the
dropout rate, the outcome measurement, and the type of analysis (eg,
intention-to-treat).

Walker, E., Hernandez, A. V., & Kattan, M. W. (2008). Meta-analysis: Its strengths and limitations. Cleveland Clinic Journal of Medicine, 75(6), 431–439.

From what I understand, the I should make some scoring system for my sample of studies, and use that score as a "weight" in my meta-analytic model?
I do not like this idea. It seems to my more less objective than meta-regression
mainly because I have no criteria in my studies to make the score. (I am doing meta-analysis of ecological studies.)

May I use this as an argument in response that stratified meta-analysis will be less objective in my case?

Best Answer

Here are some suggestions for how you could respond:

  1. Given that your predictors are continuous (and artificially categorizing predictors is usually frowned upon), meta-regression seems like the right approach (and in fact, meta-regression can also deal with categorical predictors just as well as stratifying on them, so why bother?).
  2. If I understand you correctly, you entered those 4 predictors simultaneously into the model. Stratifying would either imply examining one (artificially categorized) predictor at a time (which does not consider heterogeneity that may be better accounted for by other predictors or potential confounding between the predictors) or if one were to start stratifying on combinations of predictors, the subgroups will start to get quite small. That doesn't seem like a good idea (see also the next point).
  3. How well the amount of heterogeneity in a random-effects model (or the amount of residual heterogeneity in a mixed-effects meta-regression model) is estimated depends to a great extent on the number of studies. Stratifying will lead to smaller subsets with poorer estimates of the amount of heterogeneity.

I actually discuss these issues in this article:

Viechtbauer, W. (2007). Accounting for heterogeneity via random-effects models and moderator analyses in meta-analysis. Zeitschrift für Psychologie / Journal of Psychology, 215(2), 104-121.

If you are interested and cannot get hold of a copy of the article (it's in a German journal; but the article is in English), feel free to send me an e-mail (you'll find my website linked to from my profile; e-mail address can be found there).

Related Question