Solved – Citation for Statistical test for difference between two odds ratios

confidence intervallogisticodds-ratioreferences

In a comment here, @gung wrote,

I believe they can overlap a little (maybe ~25%) & still be significant at the 5% level. Remember that the 95% CI you see is for the individual OR, but the test of 2 ORs is about the difference between them. However, if they don't overlap at all, then they are definitely significantly different, & if the 95% CI's overlap the other OR point estimate, they definitely don't.

Does anyone have citations for the above statement? A reviewer wants me to calculate if two odds ratios are significantly different to each other.

Best Answer

From your two logistic regression models, you should have parameter estimates, $\hat\beta_{11}$ and $\hat\beta_{12}$ (where the second subscript refers to the model), and their standard errors. Note that these are on the scale of the log odds and that this is better—there is no need to convert them to odds ratios. If your $N$s are sufficient, these will be normally distributed, as @ssdecontrol explained. The Wald tests that come standard with logistic regression output assume they are normally distributed, for example. In addition, since they came from different models with different data, we can treat them as independent. If you want to test if they are equal, this is simply testing a linear combination of normally distributed parameter estimates, which is a pretty standard thing to do. You can calculate a test statistic as follows:
$$ Z = \frac{\hat\beta_{12}-\hat\beta_{11}}{\sqrt{SE(\hat\beta_{12})^2 + SE(\hat\beta_{11})^2}} $$ The resulting $Z$ statistic can be compared to a standard normal distribution to compute the $p$-value.

The quote about confidence intervals is somewhat heuristic in nature (even though correct). You should not try to use that to calculate significance.