I'm doing a secondary analysis on a large sample of children from 4- to 18-years-old using logistic regression. In addition to analyzing effects of predictors (age, sex, IQ, autism severity, medical conditions) on the outcome (sleep problems) in the full sample of children, I did analyses in groups stratified by sex and age groups (early, middle, late childhood). A reviewer commented, "As complementary to stratified associations, the authors should consider presenting results from interaction tests between predictors and subgroup variables to provide the reader with relevant statistical evidence as to whether associations truly differ between these subgroups." When doing the full and stratified sample, I get main effects for most of the predictors; but when I do the analyses with interactions in the model, I only get interactions between age and IQ. Does this mean the main effects I found in the stratified groups are not valid (apart from IQ). How do I report this and respond to the reviewer? Thanks!
Solved – Stratification vs. interaction
interactionstratification
Best Answer
It sounds like you are not comparing stratified models and interaction terms in the correct way.
When you say you "get main effects" for the predictors, I take this to mean that you find a statistically significant result. The null hypothesis is that the odds ratio is 1.
Recent work has shown tried to establish that "interaction" is a stronger (causal) form of effect modification. See this paper by Vanderweele. Be cautious with interpretation, what you're doing is not a causal hypothesis, rather a post hoc analysis. We measure both effect modification and interaction with a product term. The null hypothesis is that the product term (a ratio of odds ratios) equals 1. That means that effects do not vary across strata of the effect modifier.
Stratifying a "full" analysis by a factor is the same as entering a product-term between the stratum-factor with every single parameter in the model. You can verify that by fitting this model. The interaction terms give the difference of log-odds ratios for each term in the stratified models. You will not replicate the results by entering product terms one-at-a-time into the model.
The null hypothesis of the interaction term=1 is no effect modification: the stratum-specific effects do not differ from each other. The null hypothesis of the test of main effects is that the odds ratios do not differ from 1. There can be statistically significant interaction yet a lack of precision to show the terms are different from one (interaction without association), more often you find terms differ from 1, but the terms do not significantly differ from each other (association without interaction).