Solved – What if path c isn’t significant, but paths a and b are? Indirect effect in mediation

mediationpath-model

In a classic mediation model, we have paths shown in the diagram below

,

in which the first step of testing the mediating effect of M between X and Y is that X is significantly correlated with Y (as shown in panel A in the figure).

However, I bumped into a situation where Path a and Path b are strongly significant, but not Path C. Compared to Path c, Path c' is not significant, but the coefficient is decreased.

In this case, is it still valuable to talk about the relationship among X, Y, and M?? If so, what is the best way to address this relationship in a paper? Can we claim that X has an indirect effect but not a direct effect on Y??

I am testing the same path model with three samples, $n_1 = 124, n_2 = 49, n_3 = 166$.

Best Answer

Your approach to testing mediation appears to conform to the "causal steps approach" described in the classic methods paper by Baron & Kenny (1986). This approach to mediation entails the following steps:

  1. Test whether X and Y are significantly associated (the c path); if they are not, stop the analysis; if they are...
  2. Test whether X and M are significantly associated (the a path); if they are not, stop the analysis; if they are...
  3. Test whether M and Y are significantly associated after controlling for X (the b path); if they are not, stop the analysis; if they are...
  4. Compare the direct of effect of X (the c' path--predicting Y from X after controlling for M) to the total effect of X (the c path from Step 1). If c' is closer to zero than c, and non-significant, the research concludes that M completely mediates the association between X and Y. But if c' is still significant, the researcher concludes that M is only a "partial" mediator of X's influence on Y.

I emphasize the difference between direct (c') and total effects (c) because though you wrote...

Can we claim that X has an indirect effect but not a direct effect on Y??

I think what you are actually concerned about is the legitimacy of claiming that X has an indirect, but not a total effect on Y.

The Short Answer

Yes, it is legitimate to conclude that M mediates the association between X and Y even if the total effect (c) is not significant. The causal steps approach, though historically popular, has been widely replaced by methods of testing for mediation that are more statistically powerful, make fewer assumptions of the data, and are more logically coherent. Hayes (2013) has a wonderfully accessible and thorough explanation of the many limitations of the causal steps approach in his book.

Check out other more rigorous approaches, including the bootstrapping (MacKinnon et al., 2004) and Monte Carlo (Preacher & Selig, 2012) methods. Both methods estimate a confidence interval of the indirect effect itself (the ab path)--how they do so differs between methods--and then you examine the confidence interval to see whether 0 is a plausible value. They are both pretty easy to implement in your own research, regardless of which statistical analysis software you use.

The Longer Answer

Yes, it is legitimate to conclude that M mediates the association between X and Y even if the total effect (c) is not significant. In fact, there is a relatively large consensus among statisticians that the total effect (c) should not be used as a 'gatekeeper' for tests of mediation (e.g., Hayes, 2009; Shrout & Bolger, 2002) for a few reasons:

  1. The causal steps approach attempts to statistically evaluate the presence of mediation without ever actually directly evaluating the indirect effect (the ab path, or c-c' if you prefer). This seems illogical, especially given that there are numerous easy ways to estimate/test the indirect effect directly.
  2. The causal steps approach is contingent on multiple significance tests. Sometimes significance tests work as they should, but they can be derailed when assumptions of inferential tests are not met, and/or when inferential tests are underpowered (I think this is what John was getting at in his comment on your question). Thus, mediation could be really happening in a given model, but the total effect (c) could be non-significant simply because the sample size is small, or assumptions for the test of the total effect have not been met. And because the causal steps approach is contingent on the outcome of two other significance tests, it makes the causal steps approach one of the least powerful tests of mediation (Preacher & Selig, 2008).
  3. The total effect (c) is understood as the sum of the direct effect (c') and all indirect effects (ab(1) , ab(2)...). Pretend the influence of X on Y is fully mediated (i.e., c' is 0) by two variables, M1 and M2. But further pretend that the indirect effect of X on Y through M1 is positive, whereas the indirect effect through M2 is negative, and the two indirect effects are comparable in magnitude. Summing these two indirect effects would give you a total effect (c) of zero, and yet, if you adopted the causal steps approach, you would not only miss one "real" mediation, but two.

Alternatives that I would recommend to the causal steps approach to testing mediation include the bootstrapping (MacKinnon et al., 2004) and Monte Carlo (Preacher & Selig, 2012) methods. The Bootstrapping method involves taking a superficially large number of random samples with replacement (e.g., 5000) of the same sample size from your own data, estimating the indirect effect (the ab path) in each sample, ordering those estimates from lowest to highest, and then define a confidence interval for the bootstrapped indirect effect as within some range of percentiles (e.g., 2.5th and 97.5th for a 95% confidence interval). Bootstrapping macros for indirect effects are available for statistical analysis software like SPSS and SAS, packages are available for R, and other programs (e.g., Mplus) have bootstrapping capabilities already built-in.

The Monte Carlo method is a nice alternative when you don't have the original data, or in cases when bootstrapping isn't possible. All you need are the parameter estimates for the a and b paths, each path's variance, and the covariance between the two paths (often, but not always 0). With these statistical values, you can then simulate a superficially large distribution (e.g., 20,000) of ab values, and like the bootstrapping approach, order them from lowest to highest and define a confidence interval. Though you could program your own Monte Carlo mediation calculator, Kris Preacher has a nice one that is freely available to use on his website (see Preacher & Selig, 2012, for accompanying paper)

For both approaches, you would examine the confidence interval to see if it contains a value of 0; if not, you could conclude that you have a significance indirect effect.

References

Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173-1182.

Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York, NY: Guilford.

Hayes, A. F. (2009). Beyond Baron and Kenny: Statistical mediation analysis in the new millennium. Communication Monographs, 76 408-420.

MacKinnon, D. P., Lockwood, C. M., & Williams, J. (2004). Confidence limits for the indirect effect: Distribution of the product and resampling methods. Multivariate Behavioral Research, 39, 99-128.

Preacher, K. J., & Selig, J. P. (2012). Advantages of Monte Carlo confidence intervals for indirect effects. Communication Methods and Measures, 6, 77-98.

Shrout, P. E., & Bolger, N. (2002). Mediation in experimental and nonexperimental studies: New procedures and recommendations. Psychological Methods, 7, 422-445.

Related Question