What is an appropriate way to test whether two logistic regressions are significantly different from one another?
Essentially, I have two similar logistic regressions made from two different sets of data, and one made from the two data sets combined. I'd like to be able to perform further analyses on the whole data set using the regression made from combining the data sets, but first I have to be sure that the relationship is not significantly different from either of the two original regressions. I'd also like to know if the two separate original regressions are different from one another. Is there a straightforward way to do this?
Thanks!!
Best Answer
You could test the differences between the two models by fitting one logistic regression over both data sets using a dummy variable to distinguish observations from each population. Then you could use interaction terms to simultaneously fit the two models.
Create a dummy variable Pop, setting the value equal to zero for observations from the first data set and to one for the second data set. Then, fit a logistic regression including the population dummy as a predictor and include interactions between the population dummy and all other predictors.
For example if you had one predictor $X_1$, you would fit the following model:
$$ logit(p_i) = \beta_0 + \beta_1 \cdot Pop + \beta_2 \cdot X_1 + \beta_3 \cdot Pop \cdot X_1 $$
Then you can test whether constraining $\beta_1$ and $\beta_3$ to zero significantly worsens the fit of the model using a likelihood ratio test (LRT). These two variables give the difference in the intercept and slope coefficients in the linear prediction equation when Pop is one versus zero; if you can set them to zero and not significantly worsen the fit of your model, it suggests you can use the same regression model for both data sets.
You could also just check individual significance of the interaction term coefficients for example by Wald test stats, but I think that comparing models with and without all interaction terms at once via an LRT sounds closer to what you want. Also, likelihood ratio tests are generally preferred to Wald tests.