Solved – Weighting results in a Likert survey

likertsurveyweighted mean

I have very little (i.e. high school level) stats training, so forgive me if anything in here doesn't make sense.

A team at my work has performed a training exercise for about 250 employees and has conducted a survey afterwards about how the training has affected their day-to-day work. The survey comprises about 10 statements about the training, each with 6 possible responses (Strongly Disagree", Disagree, Slightly Disagree, Slightly Agree, Agree, and Strongly Agree).

I have been tasked with figuring out how effective different aspects of the training were, based on the survey results.

The initial approach suggested was to count the percentage of Agree or Strongly Agree answers for each question and use this as an effectiveness measure.

However, I've been thinking that if somebody "strongly" agrees or disagrees with a given statement, that their opinion should be weighted more heavily (since they have formed a definite opinion on the subject), and if somebody only "slightly" agrees or disagrees, their opinion should not be as heavily weighted (since they are pretty neutral about the subject).

So, for instance, each "slightly" answer should be counted as 0.5 of a response and each "strongly" answer should be counted as 1.5 responses.

Is there any precedence for this sort of analysis, or am I just overcomplicating things?

Best Answer

The trouble with weighting is that your results will be arbitrary. For example, if you organize your responses on a 1-6 scale (1 being strongly disagree and 6 being strongly agree), then you're saying that the "distance" between a 1 and a 2 is the same as the "distance" between a 2 and a 3. (Here I use "distance" to indicate difference or the gap between what one number represents and another number represents.)

What I would suggest is, depending on your analysis, looking into an ordered model of some sort. The ordering indicates that StD < D < SlD < SlA < A < StA, but doesn't specify how large the distance is between any two options. I prefer an "ordered logit model" and that should suffice for your analysis if you have a large enough sample (which it appears that you do). This will also let you see how other factors affect their response on the survey, if you have that sort of information (i.e. gender, time with the company, department, etc.) available.

In broad strokes, ordered logit is going to be a fancy regression method that works with categorical data that has an order but isn't necessarily equally spaced out. Regression (as you may remember) is a way to measure the association between two variables by saying if one variable changes by $X$ amount, we expect the other variable to change by $Y$ amount. (I know you haven't taken a stats class recently, so hopefully this elucidates some of the ideas. There should be information online about how to conduct this sort of analysis.)

1. How do I input them in SPSS?

You can open an Excel file in SPSS. Use the standard file open option, and select file type = *xls. Try to ensure that the first row has the variable names.

2. How do I work out the frequency of replies for each recipient?

Do you mean the frequency of responses for each question?
Check out the menu Descriptive Statistics - Frequencies

3. How do I work out frequency of replies i.e agrees/disagrees etc for each group?

Check out Descriptive Statistics - Crosstabs

4. How can I rank each individual question (12 of them)? Remember, there are 3 individual statements to each question.

Rank them in terms of what?
If you intend to rank each question in terms of their mean (e.g., on a one to five scale). One way would be to run Descriptive Statistics - Descriptives and get the mean for each item. Then copy and paste the table of item means into Excel and sort by the Mean column.

5. How do I compare UK architects to US architects to show congruence or not?

Check out Descriptive Statistics - Explore; you could also look at some of the compare mean options.

6. How would show correlation between the two groups UK and US?

These are different participants so I don't know what you mean by asking for correlations.

7. Will SPSS develop graphs etc for me showing frequency or correlation?

Yes, it will.
Just have a play around with the Graphs menu (e.g., Legacy - Scatter or Legacy - Bar)

General Suggestions

It sounds like you need a basic book explaining how to use SPSS. A good one is the SPSS Survival Manual. I also wrote a 120 page PDF Introduction to SPSS several years back which explains all the things mentioned above with examples.

Solved – How to determine if two survey questions are independent

I agree with @rolando2's suggestion that Spearman's/Kendall's might be better suited. In general doing this by hand is just inconvenient but if you want to have a look at this excellent Khan Academy clip that shows exactly how to do a $\chi^2$ test.

I suggest you use some software, my current favorite is R together with RStudio as your IDE

First create your dataset, preferably in a spreadsheet and then import it but you can also create the data in R:

my_question1 <- c(1, 1, 3, 1, 4, 3, 1, 4, 4, 3, 2)
my_question2 <- c(1, 2, 4, 3, 3, 5, 2, 5, 5, 4, 1)

Then the $\chi^2$ test

chisq.test(my_question1, my_question2)

If you have a cell with few outcomes (5 or less) you should use Fisher's exact test:

fisher.test(my_question1, my_question2)

For the Spearman method use:

cor.test(my_question1, my_question2, method="spearman")

And for the Kendall use:

cor.test(my_question1, my_question2, method="kendall")

Hope this helped