I recently conducted a online survey and are now having trouble deciding for the right analysis method.
For example: I wanted to see if there is a correlation between Factor A and Factor B.
However in my survey I did not ask the participants to directly rate Factor A and Factor B, but rather asked multiple subquestions for these two factors which were rated from 5(Strongly Agree) to 1 (Strongly Disagree). I created a sum score for item A and a sum score for item B by summing up the values of each subquestion and calculated the mean of the sumscores to measure them.
So my question ist: Which method would be best to use here? I want to test if e.g. people with a higher score in Factor A would be more likely to decide for Factor B.
I actually planned using Spearman or ANOVA depending on the number of independent variables, but I am not sure if that would really be the correct way to measure the correlation.
Best Answer
Since you have a 5-level Likert scale, rather than encoding them as:
$$1, \, 2, \, 3, \, 4, \, 5$$
You could encode them as:
$$-2, -1, -0, \; \; \, 1, \; \; \, 2 \; \; \,$$
That way, summing over a large number of questions will give you a number that is positive if someone tends to agree and negative if someone tends to disagree. More importantly, you can interpret this number the same, regardless of the number of questions summed over.
Then you could simply plot the average response on factor A against the average response on factor B per participant in a scatter plot. This will give you much more compelling evidence then just some measure of correlation or a $p$-value from an analysis.
You could add a measure of correlation to the plot. If the relationship is more or less linear, you can use the Pearson correlation. Otherwise, you can use rank correlation instead, like Spearman's rank correlation.
The question only mentions factors A and B, so I see no reason to use ANOVA (comparing any number of means) over a simple correlation coefficient.