Solved – Is it appropriate to treat n-point Likert scale data as n trials from a binomial process

Question

I've never liked how people typically analyze data from Likert scales as if error were continuous & Gaussian when there are reasonable expectations that these assumptions are violated at least at the extremes of the scales. What do you think of the following alternative:

If the response takes value $k$ on an $n$-point scale, expand that data to $n$ trials, $k$ of which have the value 1 and $n-k$ of which have the value 0. Thus, we're treating response on a Likert scale as if it is the overt aggregate of a covert series of binomial trials (in fact, from a cognitive science perspective, this is actually an appealing model for the mechanisms involved in such decision making scenarios). With the expanded data, you can now use a mixed effects model specifying respondent as a random effect (also question as a random effect if you have multiple questions) and using the binomial link function to specify the error distribution.

Can anyone see any assumption violations or other detrimental aspects of this approach?

Answer 1

I don't know of any articles related to your question in the psychometric literature. It seems to me that ordered logistic models allowing for random effect components can handle this situation pretty well.

I agree with @Srikant and think that a proportional odds model or an ordered probit model (depending on the link function you choose) might better reflect the intrinsic coding of Likert items, and their typical use as rating scales in opinion/attitude surveys or questionnaires.

Other alternatives are: (1) use of adjacent instead of proportional or cumulative categories (where there is a connection with log-linear models); (2) use of item-response models like the partial-credit model or the rating-scale model (as was mentioned in my response on Likert scales analysis). The latter case is comparable to a mixed-effects approach, with subjects treated as random effects, and is readily available in the SAS system (e.g., Fitting mixed-effects models for repeated ordinal outcomes with the NLMIXED procedure) or R (see vol. 20 of the Journal of Statistical Software). You might also be interested in the discussion provided by John Linacre about Optimizing Rating Scale Category Effectiveness.

The following papers may also be useful:

1. Wu, C-H (2007). An Empirical Study on the Transformation of Likert-scale Data to Numerical Scores. Applied Mathematical Sciences, 1(58): 2851-2862.
2. Rost, J and and Luo, G (1997). An Application of a Rasch-Based Unfolding Model to a Questionnaire on Adolescent Centrism. In Rost, J and Langeheine, R (Eds.), Applications of latent trait and latent class models in the social sciences, New York: Waxmann.
3. Lubke, G and Muthen, B (2004). Factor-analyzing Likert-scale data under the assumption of multivariate normality complicates a meaningful comparison of observed groups or latent classes. Structural Equation Modeling, 11: 514-534.
4. Nering, ML and Ostini, R (2010). Handbook of Polytomous Item Response Theory Models. Routledge Academic
5. Bender R and Grouven U (1998). Using binary logistic regression models for ordinal data with non-proportional odds. Journal of Clinical Epidemiology, 51(10): 809-816. (Cannot find the pdf but this one is available, Ordinal logistic regression in medical research)