McNemar-Bowker Test – Comparison of Two Measures Explained

categorical datamcnemar-testrepeated measures

I am comparing two versions of a survey (an original, and a modified) and was wondering if the McNemar-Bowker test is appropriate.

The survey(s) ask respondents to indicate whether they've had an experience (in this case, unwanted sexual contact) since the age of 14. Respondents have four answer choices to indicate the frequency which they've had a certain experience, coded the following way: 

  "never"
  "once"
  "twice"
  "three or more times"

My hypothesis is that participants will indicate an increased number of experiences when given the modified version, as compared to the original. In this instance I am looking at the rates which men admit to having perpetrated acts of unwanted sexual contact; with the hypothesis that the modified version will ellicit increased rates of admitted perpetration compared to the original version.

My data do in fact show an increase in male admittal-rates on the modified version.

 MALE ORIGINAL     (n)                      MALE MODIFIED     (n)
 "Never" =         281                      "Never" =         270
 "Once" =           18                      "Once" =           18
 "Twice" =           7                      "Twice" =           8
 "Three +" =         7                      "Three +" =        17
  Total admittal =  32                       Total admittal =  43
  Total n =        313                       Total n =        313

So, now i want to test for significant difference between response rates on these two surveys to see if the increase of admittal on the modified version is statistically significantly higher. This is a repeated measures design, so each participant took both survey versions.

I am wondering if the McNemar-Bowker test is appropriate for my data (above)? My plan is to arrange the table as such: 

                                 ORIGINAL SURVEY


                      |"Never" | "Once" | "Twice" | "Three +" | TOTAL |
                      |________|________|_________|___________|_______|
               "Never"|  261   |   7    |    2    |     0     |  270  |
                      |________|________|_________|___________|_______|
                "Once"|   12   |   5    |    1    |     0     |   18  |
                      |________|________|_________|___________|_______|
               "Twice"|    3   |   4    |    1    |     0     |    8  |
                      |________|________|_________|___________|_______|
             "Three +"|    5   |   2    |    3    |     7     |   17  |
                      |________|________|_________|___________|_______|
                TOTAL |  281   |   18   |    7    |     7     |  313  |
                      |________|________|_________|___________|_______|

MODIFIED
SURVEY

When I run the above analysis in SPSS crosstabs, I get a McNemar-Bowker chi-square value of 13.316, df = 6, p = 0.038. So my questions are:

  1. Is the McNemar-Bowker appropriate for my data and hypothesis?
  2. With
    the resulting p value of 0.038 can I now say that the results from
    the two surveys are significantly different — with the modified
    version showing an increase in male admittal?

Best Answer

McNemar-Bowker test of symmetry of k X k contingency table is inherently 2-sided: the alternative hypothesis is undirected. So, in general case it cannot be used to test a one sided alternative that subdiagonal frequencies are larger/smaller than superdiagonal frequencies. But since in your case the differences are consistently in favour of subdiagonal frequencies you can use the test for the directional inference.

The Bowker test is chi-square asymptotic-based and hence is for "large sample" - I've read somewhere (sorry don't remember where, so I'm not quite sure) that the sum in any two symmetric cells, if it is not 0 (the test ignores 0-0 cell pairs altogether), should be at least 10. Clearly, this isn't your case - you have only one pair of symmetric cells with the large sum. There exists an exact version of the test (see) but not in SPSS. But you can bypass the problem if you merge "Once", "Twice", "Three+" categories. Then you'll have the dichotomous case for which Bowker test becomes McNemar test with exact p-value easily computed (SPSS does it).

You might want also to consider some alternative tests of symmetry of a contingency table. Because it is questionnable whether your inquiry is isomorphic to what McNemar-Bowker tests. It tests if every off-diagonal cell is equal (in population) to the cell symmetric to it. Might it be that comparing the subdiagonal and the superdiagonal sums is more apt here?

Related Question