Solved – McNemar Test: Reporting Chi-Square

hypothesis testingmcnemar-testspss

I have performed a McNemar test for paired binomial variables in SPSS to see if the success rate(score 1) differs between the two conditions (the two variables: the before and after treatment variables). When I run the McNemar test using the cross-tabs function in SPSS it does not report a test-value, only the Exact Sig (2-sided), which in my case is .000, so significant. It also notes that the binomial distribution is used to determine this.

However, when I run the McNemar test using the nonparametrics tests –> related samples function, the Chi-Square is also reported, 58.061, AsympSig .000. In the end the most important thing is that there is a significant change from before to after the treatment. However, I am wondering if I should also report the Chi-Square value. The first p-value is based on the binomial distribution so I guess reporting the Chi-Square does not make sense. On the other hand, why not just report the Chi-Square and the accompanying p-value of the second method?

I hope someone has some insight on this.Thanks!

Best Answer

It may help you to read my answers to What is the difference between McNemar's test and the chi-squared test, and how do you know when to use each? here, and here. The short version is that McNemar's test is actually a binomial test of whether the two off-diagonal cell counts (often denoted cells b & c) diverge from an expected null ratio of $1$ to $1$. However, it is also possible to test them as $(b-c)^2/(b+c)$. The latter quotient would be distributed as a chi-squared random variable. In other words, what SPSS seems to be reporting in your "nonparmetric" output is the test statistic for McNemar's test, not a chi-squared test of independence.

As @ttnphns notes in the comments, the two possible ways of conducting McNemar's test are asymptotically equivalent. When you sample is small, though, the binomial will be more accurate.

With regard to what you should report, that would depend on which method you used. I would report the full $2\times 2$ table, or at least the b & c cell counts, and then either the observed proportion $b/(b+c)$ with the binomial $p$-value, or the chi-squared test statistic with its corresponding $p$-value.