My dataset consists of 120 observations, and 7 observed variables. I'm using 3 factors (result from parallel analysis and theory). All 7 variables load relatively well (2 in factor 1, 2 in factor 2, 3 in factor 3). The Tucker Lewis Index = 1.034 and CFI = 1.004, which looks OK. However, I'm getting RMSEA and RMSR values of zero and I'm not sure what to make of this. Any help is appreciated.
Solved – How to interpret RMSEA = 0 and RMSR = 0 in exploratory factor analysis
factor analysis
Related Solutions
I will just address question 2. I have some doubts about how well the author knows his subject if he really said it the way you have presented it. PCA is applied to the sample just like EFA and CFA. It simply takes a list of n possibly related factors looks at how the points (samples) scatter in n-dimension space and then gets the first principal component as the linear combination that explains more of the variability in the data than any other linear combination. Then the second looks at orthogonal directions to the first to find theone out of those that explains the most of the remaining variability and so on with the 3rd and 4th. So sometimes one can take just 1-3 components to describe most of the variation in the data. That is why factor analysis and principal componet analysis are described according to 1 and 2 in your statement.
Both the KMO and Bartlett’s test of sphericity are commonly used to verify the feasibility of the data for Exploratory Factor Analysis (EFA).
Kaiser-Meyer Olkin (KMO) model tests sampling adequacy by measuring the proportion of variance in the items that may be common variance. Values ranging between .80 and 1.00 indicate sampling adequacy (Cerny & Kaiser, 1977).
Bartlett’s test of sphericity examines whether a correlation matrix is significantly different to the identity matrix, in which diagonal elements are unities and all off-diagonal elements are zeros (Bartlett, 1950). Significant results indicate that variables in the correlation matrix are suitable for factor analysis.
The remaining four measures of fit can be used in EFA (see Aichholzer (2014) for an example) but in my experience, these fit measures are more commonly applied as part of the Confirmatory Factor Analysis and Structural Equation Modelling, in which you test whether your proposed model conforms to its expected factor structure, just like in the second paper you referenced).
- This pdf by Hooper (2008): Structural Equation Modelling: Guidelines for Determining Model Fit provides a concise and straight to the point summary of each fit statistic you listed and more. As of 2019, this is indeed quite a cited article with over 7,000 citations.
Before providing a concise summary of the aforementioned fit statistics, it is worth noting that there are different classifications of fit indices, but one popular classification distinguishes between absolute fit indices and comparative fit indices.
Classification of fit indices: Absolute and Comparative
The logic behind absolute fit indices is essentially to test how well the model specified by the researcher reproduces the observed data. Commonly used absolute fit statistics include the $\chi^2$ fit statistic, RMSEA, SRMR.
In contrast, comparative fit indices are based on a different logic, i.e. they assess how well a model specified by a researcher fits the observed sample data relative to a null model (i.e., a model that is based on the assumption that all observed variables are not correlated) (Miles & Shevlin, 2007). Popular comparative model fit indices are the CFI and TLI.
The $\chi^2$ fit statistic
The $\chi^2$ measures the discrepancy between the observed and the implied covariance matrices.
The $\chi^2$ fit statistic is very popular and frequently reported in both CFA and SEM studies.
However, it is notoriously sensitive to large sample sizes and increased model complexity (i.e. models with a large number of indicators and degrees of freedom). Therefore, the current practice is to report it mostly for historical reasons, and it rarely used to make decisions about the adequacy of model fit.
The RMSEA
The Root Mean Square Error of Approximation (RMSEA) provides information as to how well the model, with unknown but optimally chosen parameter estimates, would fit the population covariance matrix (Byrne, 1998).
It is a very commonly used fit statistic.
One of its key advantages is that the RMSEA calculates confidence intervals around its value.
Values below $.060$ indicate close fit (Hu & Bentler, 1999). Values up to $.080$ are commonly accepted as adequate.
The SRMR
The Standardized Root Mean Residual (SRMR) is the square root of the difference between the residuals of the sample covariance matrix and the hypothesized covariance model.
As SRMR is standardized, its values range between $0$ and $1$. Commonly, models with values below $.05$ threshold are considered to indicate good fit (Byrne, 1998). Also, values up to $.08$ are acceptable (Hu & Bentler, 1999).
The CFI and TLI
Two comparative fit indices commonly reported are the Comparative Fit Index (CFI) and the Tucker Lewis Index (TLI). The indices are similar; however, note that the CFI is normed while the TLI is not. Therefore, the CFI’s values range between zero and one, whereas the TLI’s values may fall below zero or be above one (Hair et al., 2013).
For CFI and TLI values above .95 are indicative of good fit (Hu & Bentler, 1999). In practice, CFI and TLI values from $.90$ to $.95$ are considered acceptable.
Note that the TLI is non-normed, so its values can go above $1.00$
EDIT:
Further to the aforementioned information, Hoyle (2012) provides an excellent succinct summary of numerous fit indices. This table includes, for example, information on the indices' theoretical range, sensitivity to varying sample size and model complexity. Note that, in contrast to the indices introduced above, a great number of other indices exist, as illustrated in Hoyle's table. Yet, the frequency of their use is decreasing for various reasons. For example, RMR is non-normed and thus it is hard to interpret. Here these indices are shown below simply for everyone's general awareness, i.e. the fact that they exist, who developed them and what their statistical properties are.
References
Aichholzer, J. (2014). Random intercept EFA of personality scales. Journal of Research in Personality, 53, 1-4.
Bartlett, M. S. (1950). Tests of significance in factor analysis. British Journal of Statistical Psychology, 3(2), 77-85.
Byrne, B.M. (1998). Structural Equation Modeling with LISREL, PRELIS and SIMPLIS: Basic Concepts, Applications and Programming. Mahwah, NJ: Lawrence Erlbaum Associates.
Cerny, B. A., & Kaiser, H. F. (1977). A study of a measure of sampling adequacy for factor-analytic correlation matrices. Multivariate Behavioural Research, 12(1), 43–47.
Hair, R. D., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2013). Multivariate data analysis. Englewood Cliffs, NJ: Prentice–Hall.
Hooper, D., Coughlan, J., & Mullen, M. R. (2008). Structural equation modeling: Guidelines for determining model fit. Electronic Journal of Business Research Methods, 6(1), 53-60.
Hoyle, R. H. (2012). Handbook of structural equation modeling. London: Guilford Press.
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modelling, 6(1), 1–55.
Miles, J. & Shevlin, M. (2007). A time and a place for incremental fit indices. Personality and Individual Differences, 42(5), 869-74.
Best Answer
It appears that you have very good fit. Any time that chi-square is less than df, then RMSEA will be zero.
But the value of RMSR is a bit troubling. Is it exactly zero? I would be a little suspicious, and check this out further. Can you get the predicted correlation matrix, and compare it to what you have?
CFI is also troubling. CFI cannot go above 1.00 (normally). (TLI can - that's OK.)