What test for significance is best to use to test the difference between the change of a categorical variable in a control group and the change in a pilot group?
The variable had three categories (e.g. on-street parking, off-street parking, and garage parking). We want to see if the change of the percent of people who utilized each of these different types of parking in the our control area was statistically different than the change of the percent of people who utilized each in out pilot area.
As an example:
Parking on-street:
Pilot before = 209/737 = 28%
Pilot after = 145/754 = 19%
Net percentage point change = -9%
Control before = 297/713 = 42%
Control after = 242/716 = 34%
Net percentage point change = -8%
Was the change in the pilot area statistically different from the change in the control area?
I do not think I can simply compare the 'after' for each because the 'before' for the two groups was statistically different – they did not start out the same so I believe I can only compare the change within each group.
I used the two-tailed, two-proportion z-test for within group differences.
Best Answer
Does your categorical variable only have 2 levels? or more?
If only 2 levels then you could fit a logistic regression (or generalized linear mixed model) with effects for group (control/pilot), time (before/after), and subject (this would be the random effect if doing the mixed model) plus the interaction between group and time. The interaction term would be the main one of interest in seeing if the change was different between the 2 groups. Conditional logistic regression may be a compromise between the above 2 methods.
If more than 2 levels you could either combine groups until you only have 2 and do the above, or use a multinomial regression (but this is much more complicated with the design you describe).