I am looking to compare individual likert scale items.
I measured respondent's level of agreement (in a 5 point-scale) for several items (e.g. from 1 to 5 how much is A competent to treat your condition? how much is B? C? and so on..- 11 items in total). I want to analyse if the scores for each statement are significantly different?
I am considering the data as ordinal and as such using non-parametric analysis. So far what I did was a Friedman test (which should indicate if there is agreement between k set of ranking) (p<0.001). So indeed the distribution of the several scores (of the several items) are different. But I also wanted to know which items are statistically significanly different. So I did a Kendall correlation for all items.
- Is this the right approach?
- Can anyone let me know what is the best way to compare likert-scale items?
Best Answer
As I understand your question, Friedman's test is a reasonable test to use.
You are treating the items as treatments, groups, or "subjects"†. And you are treating respondents as blocks or raters. And you have unreplicated complete block design.
But you might want to reconsider your choice of post-hoc test. If you want to compare among the items, you want to use an accepted post-hoc test for Friedman's. One common test is often called Conover test. Another is often called Nemenyi test.
These are available in software packages.
For some references, and other tests, see the descriptions for functions beginning with "frd" in this R package.
Using correlations will not make sense as a post-hoc test for how you are using the Friedman test, though they may be interesting to answer other questions.
† Sorry, it's not my terminology, but confusingly the term "subject" is used for the treatments, especially when Kendall's W, an effect size statistic for Friedman's test, is being discussed.