My data are in SPSS. I teach in a nursing program.
During their junior year, 68 students participated in a simulation scenario for heart failure. They were by rated by observers (0/1, did not do/did do) on 102 items, thus a student's score could range from 0-102 (actual range was 26-86 with a mean of 57.41 and SD = 15.49). Higher scores indicated better performance.
Their senior year, these same 68 students participated in a pneumonia scenario. Once again, they were rated by observers but the rating scale had 91 items. Actual range of scores was 0-91 with a mean of 52.48 and SD = 14.6.
I want to compare student performance from jr. to sr. year to see if it improved. Note that the mean score sr. year is lower but that may be because the rating scale had 11 fewer items.
Since the scales have a different number of items, I am thinking I need to standardize the scores. Thus I had SPSS calculate z-scores for the total score for the pneumonia and heart failure rating scales. Where I am stuck is what to do next? Calculating a paired samples t-test using the z-score is not working, I am assuming because the scores are standardized to have a mean of 0 and SD of 1.
I am sure there is a simple solution but I am stumped. After looking at this all afternoon I can't come up with a solution. Therefore, thanks in advance to anyone who can give me an answer/suggestion of what to do. Thanks!
Best Answer
$Z$-scores are definitely NOT the answer. You would need to make too many assumptions about the meaning on standard deviations. You have both time and a change in the disease being simulated so it's not clear on what you mean by 'improved. But to your original question you would need to have a clear and meaningful way to calibrate the scales to each other. The simplest approach is to divide by the number of items but this would need a lot of though. Overall your design is confounded and interpretation will be difficult