Solved – Which statistical test should I use – likert scales and multiple categorical groups

hypothesis testinglikertmultivariate analysisstatistical significance

I would like to run a test to see if the difference in scores on a 5-point likert scale (Strongly Disagree to Strongly Agree) is statistically significant between 6 different and distinct groups.

I have seen some debate about whether to regard the scales as ordinal or interval data, so I'm unsure with how to do this.

Thanks,
Chloe

Best Answer

It is usually advised to treat responses from individual Likert items as ordinal data. It requires an additional assumption about the spacing of the response categories in order to treat the responses as interval. Also, data from Likert items are likely to not approximate a continuous distribution for tests that make this assumption.

Ordinal regression is a flexible approach for analyzing ordinal data. Some modern software makes ordinal regression relatively easy.

For simple designs, some nonparametric tests, such as Mann-Whitney and Kruskal-Wallis, appear to behave well with ordinal data.

An appropriate post-hoc test for Kruskal-Wallis is Dunn test (1964).

There are a few appropriate effect size statistics for K-W. I think my favorite right now is pairwise Vargha and Delaney's A. VDA can be interpreted as the probability of an observation from one group being larger than an observation from the other group. I think this is relatively easy to understand relative to other choices for effect size statistics.