Difference between Multilevel I^2 and ICC

intraclass-correlationmeta-analysismetaformultilevel-analysis

I have been using the excellent metafor package for a work-related project and for some reason there's a concept that I can't wrap my head around.

In his guide (https://www.metafor-project.org/doku.php/analyses:konstantopoulos2011?s[]=multilevel) Wolfgang outlines how to calculate the ICC for three-level meta-analytic models based on the following formula:

This can be interpreted as the correlation in "true" effects within a cluster.

In another guide Wolfgang describes how you can calculate a multilevel version of I^2 and also parse this into it's components (between and within) through the following.

100 * model$sigma2 / (sum(model$sigma2) + (model$k-model$p)/sum(diag(P)))

This produces two values (one for within cluster, and a second value for between cluster), and represents the proportion of variance explained due to heterogeneity within each level, respectively.

However, people often refer to ICC's as a measure of the variance explained/accounted for (for example by multiplying by 100 to get the value in percentage units), which confuses me a little when it comes to differences in the interpretation.

My questions are:

  1. Is it correct to say that the ICC represents the similarity between two observations from the same cluster?

  2. What is the difference between the within cluster I^2 value and the ICC? If I^2 value for within cluster represents the proportion of variance explained by heterogeneity, then how is the ICC different when expressed as a %? Is this not also a measure of the proportion of the total variance explained?

Best Answer

  1. The ICC in the example/model you linked to (and the correct equation is $ICC = \frac{\sigma^2_1}{\sigma^2_1 + \sigma^2_2}$, that is, we need to plug the variances into the equation, not the SDs) represents the correlation between the underlying true effects from the same cluster.

  2. The 'between-cluster' $I^2$ is indeed quite similar and is given by $I^2 = \frac{\sigma^2_1}{\sigma^2_1 + \sigma^2_2 + \tilde{v}}$, where $\tilde{v}$ can be interpreted as the 'typical' sampling variance. This can be interpreted as the amount of total variance (the sum of the between- and within-cluster heterogeneity and the amount of sampling variability) that is due to between-cluster heterogeneity. One could also interpret this as the correlation between two observed effects from the same cluster (assuming that these two observed effects have sampling variances equal to $\tilde{v}$).

  3. The 'within-cluster' $I^2$ is $I^2 = \frac{\sigma^2_2}{\sigma^2_1 + \sigma^2_2 + \tilde{v}}$. It indicates how much of the total variance is due to within-cluster heterogeneity. This one doesn't lend itself to a correlation interpretation.

Related Question