I have seen in some papers that they have reported ICC as well as Cronbach's alpha. Does Cronbach's alpha provide more information about the reliability of a measured variable (e.g. heart rate at two different occasions) that ICC cannot? I presumed that since Cronbach's alpha is usually higher than ICC why one should report it as well?
Could someone elaborate on the difference between ICC and Cronbach's alpha?
Many thanks in advance
Best Answer
There are four main formulations of the two-way intraclass correlation coefficient with different names according to different conventions. Whether these are considered "mixed effects" or "random effects" models does not affect their calculation (it only affects their interpretation), so I will lump the two together here based on functional equivalence and ignore that they are different in theory.
The differences between these formulations are derived from whether they are estimating the reliability of a single measure (e.g., one item, rater, or measurement occasion) or the average of all measures (e.g., all items, raters, or occasions), and whether they are quantifying consistency (i.e., allowing each measure to have its own mean) or agreement (i.e., requiring all measures to have the same mean). All other things being equal, average measures ICCs tend to be higher than single measures ICCs, and consistency ICCs tend to be higher than agreement ICCs.
If you have to choose a single ICC formulation, pick the one that most closely matches how you will use the measure(s): if you will use a single measure, then use one of the single measures formulations but if you will use the average of all measures then use one of the average measures formulations. Similarly, if your application is okay with different measures having different means, then you can use one of the consistency formulations but if your application needs all measures to have the same mean then use one of the agreement formulations. In practice, however, you can calculate all four and learn more about your data by examining the extent to which they differ in magnitude.
Cronbach's alpha coefficient is functionally equivalent to the third formulation listed above: the average measures consistency formulation or ICC(C,k). If you don't believe me, ask SPSS to output the two-way consistency ICCs and compare the alpha value it gives you to the average measures ICC it gives you; they will be the same. Cronbach's alpha is probably included in the SPSS output alongside the ICC values because alpha is a popular measure in many fields; you definitely don't "have to" report both and, indeed, reporting it alongside ICC(C,k) would be redundant. I wouldn't rely on the SPSS defaults (in this case or many others) to guide you toward statistical "best practice."
References
McGraw, K. O., & Wong, S. P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46.
Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428.