I have one interval variable on a 1-25 point interval scale, i.e. placement test scores where 25 is the highest competence score (data not normally distributed), and six different ordinal variables on a 1-7 likert scale (where 7 is the highest score) in my data set. There are 12 participants (12 interval test scores) and 6 raters (each scored 6 different ordinal variables per participant).
Based on what I've read, I've decided to run the Spearman's correlation between the interval and each ordinal variable (i.e. six different correlation tests between the interval variable and each ordinal variable) in SPSS but would it be one-tailed? Or is there any other test I should run to test the level of agreement between the raters in my study?
Thank you for your time and consideration in reading this question!
Best Answer
It is not completely clear what you end goal is, are you interested in the agreement between raters? or in using the ratings to predict the test score? or the test score to predict the rater score? or something else?
There are more options than just correlation for looking at inter-rater agreement, including values that can be calculated on multiple raters, see this Wikipedia article for some starting reading.
When choosing between a one-tailed or two-tailed test, the decision is based on the science and the question, not the data. If you a priori believe that if there is a relationship then it will only be in one direction (or that a relationship in the other direction will have the same results as no relationship) then a one-tailed test is appropriate. If a relationship in either direction is of interest then you should do a two-tailed test, or if you will create a confidence interval and don't want it going to infinity in one direction, then you should do a two tailed test. But note that not all statistics needs to be specifically a test.