I am working on a survey analysis looking at the effect of an intervention with three components (each having multiple sub-parts). Questions ask respondents how frequently they used each component. Two of the questions have scales ranging from 1 to 5 (never/rarely/often/almost always/always), while the other question has a scale ranging from 1 to 4 (never/1-2 times/3-4 times/5 or more times).
Here are my questions:
-
Is it okay to sum all three questions together despite being on different scales? If not, how should I remedy the scale differences given that they are all measuring the same underlying construct (how the overall intervention was adopted)?
-
Each question has a different number of sub-parts (Q1 has 4, Q2 has 5, and Q3 has 6) b/c the number of items in each part of the intervention varies. Therefore, since the answer scale of Q1 is 1 to 5 and it has 4 sub-parts, the theoretical sum for any respondent could range from 4 (lowest) to 20 (highest), but for Q2, the range is obviously different (5 to 25). Q3 is on a different range (6 to 24) because the answer scale range is 1 to 4 and it has 6 sub-parts. How can I standardize the ranges? Do I even need to?
I hope these questions make sense and thanks in advance for any help! It is greatly appreciated!
Best Answer
In the wonderful book Statistics as Principled Argument Robert Abelson notes that often, in statistics, it's not a question of what you "can" do but what you can defend sensibly.
Technically, adding Likert scales is wrong - we don't know that the intervals between levels are equal, so they are only ordinal variables and we could just as well code them 1 2 4 8 12 or even 1 1.1 1.2 1.3 4000 as the more usual 1 2 3 4 5.
But people add them all the time, because although we don't know that the intervals are equal, we can posit that they are and have sensible results. So, for your first question, think about whether it makes sense in your situation to say that a 1 in the first scale is like a 1 in the second, and so on. Certainly "never" = "never" but is "1 to 2 times" = "rarely"? Well, "rarely" is context driven; in your context, does it mean something like 1 to 2 times? (Think about the question you're asking.... "How often do planes crash?" vs. "How often are planes delayed?" the response "rarely" will mean different things!
For your second question, you could take the average value per question (if adding the questions makes sense, taking their average should, as well).