Solved – do with a modified Likert-scale questionnaire

likertsurvey

We are using a Likert scale questionnaire and need to be able to identify the rate of the students. What is the best way we identify this rate? What statistical methods are typically used?

We based our motivation questionnaire on the MSLQ (pdf) but we just have a 5-point Likert response format instead of 7 points. We also remove some of the items in the questionnaire which is not related to our study. We want to know if the students were being motivated to the strategy that we used. And the answer will be based on the questionnaire that we have given them. How can we show it through statistic?

Best Answer

The first thing to do is to look up the reference at the bottom of the questionnaire, it should contain more information about its structure and, possibly, a standard procedure to compute scores for each of the scales. Once you have these scores, the idea is to treat them as you would treat any other variable (what you should precisely do will depend on a lot of other things like the experimental design, other measures you want to use and of course the objectives of the study).

A big problem however is that such questionnaires are not supposed to be used the way you did. Ideally, a lot of thought and empirical work has gone into the design of the questionnaire and you can't arbitrarily change the response format or the set of questions and still expect it to have good psychometric properties. Also, even if the questionnaire is not particularly good, using only a subset would make your results more difficult to compare with other studies.

Importantly, in psychometric terms, a “Likert scale” is not a response format but a set of items designed to measure the same “construct” or quantity. It is usually not a single question or item, nor does it necessarily include all the items in a questionnaire. In this particular case, it seems that the MSLQ includes five distinct scales so it would not give you a single “motivation” score. In other words, in this model, motivation is not unidimensional, it has several dimensions that can vary independently of each other.

All this means that knowing “if the students were being motivated” is not a well-defined question that would be amenable to statistical analysis. Even if you could compute an overall motivation score or be satisfied with a “self-efficacy” or “intrinsic motivation” score, you still need to decide with what you want to compare it.

One approach is to include a control group in your study. Other solutions would be to compare the scores before and after an intervention or to combine before/after measures and treatment/control group in a single design. Yet another approach is to compare the scores you obtained with a “norm” for the relevant population or the results from previous published studies (but doing this directly usually requires using exactly the same questionnaire).

I am afraid I am only offering bad news instead of practical advice but if you don't have any of that, there is no way to decide if a particular score is “high” and not much that can be done with these data.

Related Question