I am doing a simple moderation analysis with one independent variable (IV), one moderator (M) and one dependent variable (DV).
Results indicate that a regression model containing both main effects (of IV and M) is non-significant.
After adding the interaction term of IVxM to the previous model, the new model remains non-significant.
However, the R-squared change IS significant, so the second model accounts for a significantly higher amount of the DV than model 1.
My question is:
Under these conditions, can we speak of having a moderation here?
After all, the main effects are non-significant and I believe to remember that my statistic prof told me a moderation only occurs if the model in general and both main effects are significant…
By the way, it seems as if my main effects are non-significant due to a cross-over interaction.
However, I am not sure how to find out if this cross-over interaction is significant (in SPSS). Does this also have to do with the R-squared change value?
Best Answer
Unless you have a large number of subjects AND the estimate effect sizes are meaningful for your domain AND you have a solid theory that would explain why this could happen (preferably not one made up after seeing data), I would not read to much into such result.
Remember, that The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant http://www.stat.columbia.edu/~gelman/research/published/signif4.pdf
Also, as an approximation, if you $N$ subjects to estimate main effect reliably, you need roughly $16 \times N$ subjects to estimate interactions reliably.