Solved – What’s the difference between concordance correlation and intraclass correlation

agreement-statisticscorrelationintraclass-correlationterminology

I understand there are various formulae for calculating an intraclass correlation, and various formulae for calculating a concordance correlation. Is there something that all intraclass correlations have in common, and all concordance correlations have in common?

Is it the case that intraclass correlations are appropriate for a particular type of research question, while concordance correlations are appropriate for a different type of research question? If so, why?

My curiosity in this matter flows out of this question, in which the calculation of a concordance correlation was suggested by the comments, and the calculation of an intraclass correlation was suggested in an answer. However, I see it as a separate question.

Best Answer

The modern definition of intraclass correlation (ICC) is a biased estimate of the fraction of the total variance that is due to variation between groups as pertains to the framework of analysis of variance (ANOVA), and random effects models.

What would we use this for? An intraclass correlation (ICC) can be a useful estimate of inter-rater reliability on quantitative data because it is highly flexible. A Pearson correlation can be a valid estimator of interrater reliability, but only when you have meaningful pairings between two and only two raters. What if you have more? What if your raters differ by ratee? This is where ICC comes in (note that if you have qualitative data, e.g. categorical data or ranks, you would not use ICC).

Lin's concordance correlation coefficient (CCC) measures agreement between two variables as a departure from perfect linearity of the $y=x$ type.

$\rho_c = 1 - \frac{{\rm Expected\ orthogonal\ squared\ distance\ from\ the\ diagonal\ }x=y} {{\rm Expected\ orthogonal\ squared\ distance\ from\ the\ diagonal\ }x=y{\rm \ assuming\ independence}}$

CCC is also an inter-rater measurement called an "agreement concordance" rather than an inter-rater "reliability". However, numerically, ICC and CCC can be quite close, sometimes differing in the third decimal place.

One notable difference between ICC and CCC is that CCC can also be used in ordinal (whole number) or nominal scales (named categories), and ICC cannot. However, ICC can be used for more than two raters, and CCC cannot.

Reference

Cohen, A., & Doveh, E. (2005). Significance tests for differences between dependent intraclass correlation coefficients (ICCs). Research in Multi Level Issues, 4, 373-420.

Solved – Inter-(-intra) class correlation coefficient or intra-(inter-) concordance coefficient in python

I see no one has answered in 3 years and question may not be relevant.

But I would like to provide answer for future users.

Firstly, in Python you can use pingouin package:

import pandas as pd

#create DataFrame
df = pd.DataFrame({'exam': [1, 2, 3, 4, 5, 6, 1, 2, 3, 4, 5, 6,
                            1, 2, 3, 4, 5, 6, 1, 2, 3, 4, 5, 6],
                   'judge': ['A', 'A', 'A', 'A', 'A', 'A',
                             'B', 'B', 'B', 'B', 'B', 'B',
                             'C', 'C', 'C', 'C', 'C', 'C',
                             'D', 'D', 'D', 'D', 'D', 'D'],
                   'rating': [1, 1, 3, 6, 6, 7, 2, 3, 8, 4, 5, 5,
                              0, 4, 1, 5, 5, 6, 1, 2, 3, 3, 6, 4]})

import pingouin as pg

#https://pingouin-stats.org/generated/pingouin.intraclass_corr.html
icc = pg.intraclass_corr(data=df, targets='exam', raters='judge', ratings='rating')

icc.set_index('Type')

It returns a dataframe with lots of different ICC computed (ICC1, ICC2, ICC2). Just read the docs and choose the right one.

Secondly, you can also do the same thing in R

library(irr)
data1 <- data.frame(A=c(1, 1, 3, 6, 6, 7, 8, 9, 8, 7),
                   B=c(2, 3, 8, 4, 5, 5, 7, 9, 8, 8),
                   C=c(0, 4, 1, 5, 5, 6, 6, 9, 8, 8),
                   D=c(1, 2, 3, 3, 6, 4, 6, 8, 8, 9))

icc(data1, model = "oneway", type = "consistency", unit = "single")

Don't forget to read the package docs.

Best Answer

Related Solutions

Solved – How to test whether ICCs from different groups are significantly different

Reference

Solved – Inter-(-intra) class correlation coefficient or intra-(inter-) concordance coefficient in python

Related Question