In a survey, where students were presented various stimuli, after each stimulus they were asked to rate the following:
- Quality, on a five-point scale from "Excellent" to "Bad", let's say $Q$
- Confidence in their quality ratings, on a four-point scale from "Very Confident" to "Not confident", i.e. $C$
Here are my assumptions:
-
Obviously these samples are dependent, since they belong to exactly one stimulus and student pair (am I right assuming so?)
-
Also, the questions are very similar to the Likert scale, therefore I'm unsure of whether to classify them as ordinal or interval-based (and others seem too). I'd say they're ordinal, but many researchers in my field seem to ignore that and treat them as interval-based.
Basically, what I'd like to find out is whether these ratings are dependent on each other. So, my questions are:
- Would that data be considered interval-based or ordinal?
- Which tests can I apply here? $\chi^2$? Wilcoxon signed rank test?
I've already come up with three dimensional plots that show the counts for each pair, e.g. to say "In ten cases, users chose Bad and were Not Confident about it". Or, $count(Q_1,C_4) = 10$ … But there's nothing I can statistically prove from that alone.
Best Answer
I think there are several challenges to consider.
In terms of how to visualize, the most accurate would be to use a mosaic plot, or a stacked barplot (which are practically the same in this case, but it might be easier to find a stacked barplot in excel or SPSS than the mosaic plot).
It might also be helpful to change the likert scale to a numerical (1-5) scale, and have a boxplot of each of the 4 categories of your second question. Since boxplots are based on percentiles, the meaning of the boxplot can be somewhat consistent (depending on how the quantiles are calculated when dealing with mid points) with the type of data you present.
In terms of how to analyse, there are different questions you can ask. The simplest will be "is there a correlation between the two?", that can easily be answered using the pearson correlation on the ranking of the numerical values of your scales. This correlation will actually give you the Spearman correlation measure (the correlation of the ranks). The ranking is important for cases where you will have ties (for example, the vector: 1,2,2,4 should actually become: 1,2.5,2.5,3).
The wilcoxon test is relevant if you want to answer the question if the ranks of one measure is different than the other measure. But from your question, it doesn't sound like an interesting question. You can also use the Chi-square test for a similar question, but it's power will probably be smaller.