Suppose after an initial pre-test and a treatment, a group of 30 participants are presented with 10 sentences written in a foreign language, each containing a grammatical error.
To measure participants' performance, we examine what proportion of the 10 errors has been identified on the pre-test, and on the post-test by each participant.
Now, we have a set of 30 proportions for the pre-test, and 30 corresponding proportions for the post-test.
Question
Should one now do a "Paired-Samples t-test" between the proportions on the pre-test and the proportions on the "post-test" OR ALTERNATIVELY one should do a "significance test on the difference between the two (from pre-test, and from post-test) dependent mean proportions"?
Best Answer
I think that a paired-test should certainly be used for making comparison before and after for the same individual. However, you should consider McNemar's test for comparing proportions instead. It wouldn't be right to use paired t-test as the test is used for comparing mean and it assumes the variable is about normally distributed which may not be the case for proportions. On the other hand, McNemar's test assumes a $\chi^2$ distribution. Also, the method of estimation for variance component is also different.