Solved – Interpreting the Chi-square on a multiple group SEM with lavaan

lavaanrstructural-equation-modeling

A Chi-square (p-value) of a single group SEM is easy to interpret. It is a measure of exact fit.

When one runs a multiple group, say grouping the data by gender or by the presence of a condition like a disease, lavaan outputs a chi-square, together with the chi-square of the particular groups.

How to interpret that general multigroup chi-square? I have applied a same model to two groups, one with and the other without the diagnosis of schizophrenia. In that case the single non-significant chi-square allows me to say that schizophrenia mediates whatever relation I am evaluating in the path?

I am aware that I have to run invariance tests after the multigroup analysis, but I am curious about the particular interpretation of that Chi-square.

Best Answer

Ideally, invariance tests would culminate in a model that constrained the path between the "mediating" factor and the outcome factor to equality across groups. Global $\chi^2$ values are used to evaluate relative fit between models representing different levels of invariance.$^1$ Assuming measurement invariance has been established,$^2$ testing for mediation would be accomplished by constraining the relevant structural paths to equality across groups and conducting $\Delta\chi^2$ tests.

$^1$ Cheung and Rensvold (2002) recommend using $\Delta$CFI when testing invariance across groups. $\Delta$CFI values above .01 represent non-trivial group differences.

$^2$ People have different opinions about this. Most would agree that you need to provide evidence of weak (or metric) invariance (same factors and loadings constrained to equality across groups).

Related Solutions

Solved – multiple group model vs moderated regression

I don't know of a paper. It's the sort of thing that's pretty clear if you know about SEM, and that's why no one writes it. If anyone did write it and send it to a journal, reviewers would say "This is obvious."
1. The two methods can give identical results, if you add the correct constraints to the SEM. The SEM approach makes fewer assumptions about homogeneity of variance, so it might be preferable. To make the models equivalent you need to add constraints. (The variance of y can be different in the two groups in the SEM approach, it can't in the multilevel approach.

Here's an example (using Lavaan, in R). Everything in the regression can be seen in the lavaan output.

> library(lavaan)
> set.seed(1234)
> df <- data.frame(x = rnorm(1000))
> df$m <- c(rep(0, 500), rep(1, 500))
> df$y <- df$x + rnorm(1000) + df$m + df$m * df$x + rnorm(1000)
> 
> summary(lm(y ~ x + m + x * m, data = df))

Call:
lm(formula = y ~ x + m + x * m, data = df)

Residuals:
    Min      1Q  Median      3Q     Max 
-4.4406 -0.9659  0.0093  0.9167  4.4616 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  0.08244    0.06193   1.331    0.183    
x            1.06196    0.05991  17.727   <2e-16 ***
m            0.92634    0.08766  10.568   <2e-16 ***
x:m          1.01805    0.08815  11.548   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.385 on 996 degrees of freedom
Multiple R-squared:  0.5902,    Adjusted R-squared:  0.5889 
F-statistic: 478.1 on 3 and 996 DF,  p-value: < 2.2e-16

> 
> mod.1 <- "
+   y ~ c(a, b) * x
+   y ~~ c(v1, v1) * y  # This step needed for exact equivalence
+   y ~ c(int1, int2) * 1
+ 
+   modEff := a - b
+   mEff := int1 - int2
+ "
> 
> fit.1 <- sem(mod.1, data = df,
+                 group = "m")
> summary(fit.1)
lavaan (0.5-18) converged normally after  15 iterations

  Number of observations per group         
  0                                                500
  1                                                500

  Estimator                                         ML
  Minimum Function Test Statistic                0.499
  Degrees of freedom                                 1
  P-value (Chi-square)                           0.480

Chi-square for each group:

  0                                              0.244
  1                                              0.255

Parameter estimates:

  Information                                 Expected
  Standard Errors                             Standard

Group 1 [0]:

                   Estimate  Std.err  Z-value  P(>|z|)
Regressions:
  y ~
    x         (a)     1.062    0.060   17.762    0.000

Intercepts:
    y      (int1)     0.082    0.062    1.334    0.182

Variances:
    y        (v1)     1.910    0.085



Group 2 [1]:

                   Estimate  Std.err  Z-value  P(>|z|)
Regressions:
  y ~
    x         (b)     2.080    0.065   32.228    0.000

Intercepts:
    y      (int2)     1.009    0.062   16.294    0.000

Variances:
    y        (v1)     1.910    0.085


Defined parameters:
    modEff           -1.018    0.088  -11.572    0.000
    mEff             -0.926    0.087  -10.589    0.000

(Actually I didn't do the final step of constraining the model and testing with the anova() function, I just used the difference - that's left as an exercise for the reader, but the result will be the same p-value and no parameter estimate and standard error.)

Chi-square gives a p-value. If you don't like the p-value from the chi-square, you don't like the p-value from the regression model. It's the same estimate of the interaction effect, same standard error, same p-value, whichever method you use.
Not really. But you can, and it's easy to relax the assumption. In regression you make the assumption and you're stuck with it. In SEM the assumption is automatically tested. I added the constraint to the above model, and so it has 1 df. The chi-square test is not significant, so I don't have evidence that the assumption was violated. But you don't need to put in that constraint, so I suspect most people wouldn't.

Categorical Data – How to Model an Interaction Between a Categorical IV and a Continuous Moderator in SEM Using Lavaan in R

No, the : operator only works on observed variables. It triggers lavaan to actually calculate the product term and include it in the covariance matrix and mean vector to which the model is fitted. That cannot be what happens with a latent variable, which is not part of the observed variables' summary statistics.
Moderation is symmetric, so you could use a multigroup model, with the categorical IV as the grouping variable. Differences in the DV~MOD simple effects across groups would be moderation by FAC1. Differences in the DV~1 intercepts across groups would be the simple effect of FAC1, which could be probed by centering MOD's mean at different values. Or you might be able to use the emmeans utility in the semTools package to probe the interaction; see the ?lavaan2emmeans help-page examples. I suggest another possibility below.
- Note that using this approach would make the Fac12 variable redundant because Fac2's intercept and effects are likewise moderated by Fac1 by virtue of parameters differing across groups.
Yes, Mplus can simply use LMS estimation, but that is fraught with some restrictive assumptions. My student's PhD research (also here) has revealed that the product-indicator approach is less restrictive, and that can be implemented in lavaan (see her tutorial about using this method for invariance testing, available for download from my faculty page).

Best Answer

Related Solutions

Solved – multiple group model vs moderated regression

Categorical Data – How to Model an Interaction Between a Categorical IV and a Continuous Moderator in SEM Using Lavaan in R

Related Question