I and another coder want to assess our agreement.
Our experiment involves coding the utterances of participants. there are 17 different codes that can be assigned, and we have independently counted their frequencies.
Participant 1 may have uttered code 1: 3 times, code 2: 2 times, code 3: 1 time, and so on until code 17.
participant 2, ditto.
On top of all this, I and the other coder want to compare the frequencies we found.
So for participant 1, coder 1 and coder 2 might agree on the frequency of code 1, and not on code 3, and so on for participant 2 and 3 and 4.
I don't know of a way to ascertain the overall interrater reliability between us- is there a way? and can it be done in SPSS?
Best Answer
Actually, not in SPSS, but in R, very simply, you could use the package irr. Check this link http://www.cookbook-r.com/Statistical_analysis/Inter-rater_reliability/