I am not sure how to proceed with this CFA im doing in lavaan. I have a sample of 172 participants (I know that’s not much for a CFA) and 28 items with 7-point Likert scales that should load on seven factors. I did a CFA with „mlm“-estimators, but the model fit was really bad (χ2(df=329)=739.36; comparative fit index (CFI)= .69; standardized root mean square residual (SRMR)=.10; root mean square error of approximation (RMSEA)=.09; RMSEA 90% confidence interval (CI) = [.08, .10]).
I have tried the following:
-
bifactor model with one general method factor —> did not converge.
-
estimators for ordinal data („WLSMV“) —> Model fit: (χ2(df=329)=462; comparative fit index (CFI)= .81; standardized root mean square residual (SRMR)=.09; root mean square error of approximation (RMSEA)=.05; RMSEA 90% confidence interval (CI) = [.04, .06])
-
reducing the model by items that load low on a factor and add covariances among specific items –> Model fit: χ2(df=210)=295; comparative fit index (CFI)= .86; standardized root mean square residual (SRMR)=.08; root mean square error of approximation (RMSEA)=.07; RMSEA 90% confidence interval (CI) = [.06, .08].
Now my questions:
-
What should I do with such a model?
-
What would be statistically correct to do?
-
Report that it does fit or that it does not fit? And which of those models?
I’d be glad to have a discussion with you about this.
Here is the lavaan output of the CFA of the original model:
lavaan (0.5-17.703) converged normally after 55 iterations
Used Total
Number of observations 149 172
Estimator ML Robust
Minimum Function Test Statistic 985.603 677.713
Degrees of freedom 329 329
P-value (Chi-square) 0.000 0.000
Scaling correction factor 1.454
for the Satorra-Bentler correction
Model test baseline model:
Minimum Function Test Statistic 2461.549 1736.690
Degrees of freedom 378 378
P-value 0.000 0.000
User model versus baseline model:
Comparative Fit Index (CFI) 0.685 0.743
Tucker-Lewis Index (TLI) 0.638 0.705
Loglikelihood and Information Criteria:
Loglikelihood user model (H0) -6460.004 -6460.004
Loglikelihood unrestricted model (H1) -5967.202 -5967.202
Number of free parameters 105 105
Akaike (AIC) 13130.007 13130.007
Bayesian (BIC) 13445.421 13445.421
Sample-size adjusted Bayesian (BIC) 13113.126 13113.126
Root Mean Square Error of Approximation:
RMSEA 0.116 0.084
90 Percent Confidence Interval 0.107 0.124 0.077 0.092
P-value RMSEA <= 0.05 0.000 0.000
Standardized Root Mean Square Residual:
SRMR 0.096 0.096
Parameter estimates:
Information Expected
Standard Errors Robust.sem
Estimate Std.err Z-value P(>|z|) Std.lv Std.all
Latent variables:
IC =~
PTRI_1r 1.000 1.093 0.691
PTRI_7 1.058 0.118 8.938 0.000 1.156 0.828
PTRI_21 0.681 0.142 4.793 0.000 0.744 0.582
PTRI_22 0.752 0.140 5.355 0.000 0.821 0.646
IG =~
PTRI_10 1.000 0.913 0.600
PTRI_11r 0.613 0.152 4.029 0.000 0.559 0.389
PTRI_19 1.113 0.177 6.308 0.000 1.016 0.737
PTRI_24 0.842 0.144 5.854 0.000 0.769 0.726
DM =~
PTRI_15r 1.000 0.963 0.673
PTRI_16 0.892 0.118 7.547 0.000 0.859 0.660
PTRI_23 0.844 0.145 5.817 0.000 0.813 0.556
PTRI_26 1.288 0.137 9.400 0.000 1.240 0.887
IM =~
PTRI_13 1.000 0.685 0.609
PTRI_14 1.401 0.218 6.421 0.000 0.960 0.814
PTRI_18 0.931 0.204 4.573 0.000 0.638 0.604
PTRI_20r 1.427 0.259 5.514 0.000 0.978 0.674
IN =~
PTRI_2 1.000 0.839 0.612
PTRI_6 1.286 0.180 7.160 0.000 1.080 0.744
PTRI_12 1.031 0.183 5.644 0.000 0.866 0.523
PTRI_17r 1.011 0.208 4.872 0.000 0.849 0.613
EN =~
PTRI_3 1.000 0.888 0.687
PTRI_8 1.136 0.146 7.781 0.000 1.008 0.726
PTRI_25 0.912 0.179 5.088 0.000 0.810 0.620
PTRI_27r 1.143 0.180 6.362 0.000 1.015 0.669
RM =~
PTRI_4r 1.000 1.114 0.700
PTRI_9 0.998 0.105 9.493 0.000 1.112 0.786
PTRI_28 0.528 0.120 4.403 0.000 0.588 0.443
PTRI_5 0.452 0.149 3.037 0.002 0.504 0.408
Covariances:
IC ~~
IG 0.370 0.122 3.030 0.002 0.371 0.371
DM 0.642 0.157 4.075 0.000 0.610 0.610
IM 0.510 0.154 3.308 0.001 0.681 0.681
IN 0.756 0.169 4.483 0.000 0.824 0.824
EN 0.839 0.169 4.979 0.000 0.865 0.865
RM 0.644 0.185 3.479 0.001 0.529 0.529
IG ~~
DM 0.380 0.103 3.684 0.000 0.433 0.433
IM 0.313 0.096 3.248 0.001 0.501 0.501
IN 0.329 0.107 3.073 0.002 0.429 0.429
EN 0.369 0.100 3.673 0.000 0.455 0.455
RM 0.289 0.116 2.495 0.013 0.284 0.284
DM ~~
IM 0.530 0.120 4.404 0.000 0.804 0.804
IN 0.590 0.122 4.839 0.000 0.731 0.731
EN 0.588 0.105 5.619 0.000 0.688 0.688
RM 0.403 0.129 3.132 0.002 0.376 0.376
IM ~~
IN 0.439 0.126 3.476 0.001 0.763 0.763
EN 0.498 0.121 4.128 0.000 0.818 0.818
RM 0.552 0.122 4.526 0.000 0.723 0.723
IN ~~
EN 0.735 0.167 4.402 0.000 0.987 0.987
RM 0.608 0.141 4.328 0.000 0.650 0.650
EN ~~
RM 0.716 0.157 4.561 0.000 0.724 0.724
Variances:
PTRI_1r 1.304 0.272 1.304 0.522
PTRI_7 0.613 0.153 0.613 0.314
PTRI_21 1.083 0.199 1.083 0.662
PTRI_22 0.940 0.141 0.940 0.582
PTRI_10 1.483 0.257 1.483 0.640
PTRI_11r 1.755 0.318 1.755 0.849
PTRI_19 0.868 0.195 0.868 0.457
PTRI_24 0.530 0.109 0.530 0.473
PTRI_15r 1.121 0.220 1.121 0.547
PTRI_16 0.955 0.200 0.955 0.564
PTRI_23 1.475 0.219 1.475 0.691
PTRI_26 0.417 0.120 0.417 0.213
PTRI_13 0.797 0.113 0.797 0.629
PTRI_14 0.468 0.117 0.468 0.337
PTRI_18 0.709 0.134 0.709 0.635
PTRI_20r 1.152 0.223 1.152 0.546
PTRI_2 1.178 0.251 1.178 0.626
PTRI_6 0.942 0.191 0.942 0.447
PTRI_12 1.995 0.235 1.995 0.727
PTRI_17r 1.199 0.274 1.199 0.625
PTRI_3 0.882 0.179 0.882 0.528
PTRI_8 0.910 0.131 0.910 0.472
PTRI_25 1.048 0.180 1.048 0.615
PTRI_27r 1.273 0.238 1.273 0.553
PTRI_4r 1.294 0.242 1.294 0.510
PTRI_9 0.763 0.212 0.763 0.382
PTRI_28 1.419 0.183 1.419 0.804
PTRI_5 1.269 0.259 1.269 0.833
IC 1.194 0.270 1.000 1.000
IG 0.833 0.220 1.000 1.000
DM 0.927 0.181 1.000 1.000
IM 0.470 0.153 1.000 1.000
IN 0.705 0.202 1.000 1.000
EN 0.788 0.177 1.000 1.000
RM 1.242 0.257 1.000 1.000
Best Answer
1. Go back to Exploratory Factor Analysis
If you're getting very bad CFA fits, then it's often a sign that you have jumped too quickly to CFA. You should go back to exploratory factor analysis to learn about the structure of your test. If you have a large sample (in your case you don't), then you can split your sample to have an exploratory and a confirmatory sample.
The benefits of EFA is that it gives a lot of freedom, so you'll learn a lot more about the structure of the test than you will from only looking at CFA modification indices.
Anyway, hopefully from this process you may have identified a few issues and solutions. For example, you might drop a few items; you might update your theoretical model of how many factors there are and so on.
2. Improve the Confirmatory Factor Analysis Fit
There are many points that could be made here:
CFA on scales with many items per scale often perform poorly by traditional standards. This often leads people (and note I think this response is often unfortunate) to form item parcels or only use three or four items per scale. The problem is that typically proposed CFA structures fail to capture the small nuances in the data (e.g., small cross loadings, items within a test that correlate a little more than others, minor nuisance factors). These are amplified with many items per scale.
Here are a few responses to the above situation:
modificationindices(fit)inlavaan.General comments
So in general, if you're CFA model is really bad, return to EFA to learn more about your scale. Alternatively if your EFA is good, and your CFA just looks a little bad due to well known problems of having many items per scale, then standard CFA approaches as mentioned above are appropriate.