Solved – Omega vs. alpha reliability

psychometricsreliabilityscales

I wonder if someone can explain what is the main difference between omega and alpha reliabilities?

I understand an omega reliability is based on hierarchical factor model as shown in the following picture, and alpha uses average inter-item correlations.

enter image description here

What I don't understand is, in what condition, omega reliability coefficient would be higher than alpha coefficient, and vice versa?

Can I assume if the correlations between the subfactors and the variables are higher, the omega coefficient would also be higher (as shown in the above picture)?

Any advice is appreciated!

Best Answer

The $\omega_h$ (hierarchical) coefficient gives the proportion of variance in scale scores accounted for by a general factor (1,2), usually from a second-order factor analysis. However, if any zero-order dimensions are reflected in such scales, $\omega_h$ will be less than Cronbach's $\alpha$ (which should only be used with unidimensional scales in any case). It is only when the measurement instrument is so-called tau-equivalent (equal factor loadings but possibly unequal but uncorrelated errors) that $\alpha=\omega_h$. This was early demonstrated by McDonald. Regardless of the indicator used, low values indicate that it makes no sense to compute a sum score (i.e., to add contribution of each item score together to derive a composite score).

To sum up, correlated measurement errors, multidimensionality or unequal factor loadings make both indicators likely to diverge, with hierarchical $\omega_h$ being the reliability measure to use, following Revelle and coworkers' past work (see (1) for more discussion about that).

References

Zinbarg, R.E., Revelle, W., and Yovel, I. (2007). Estimating $\omega_h$ for structures containing two group factors: Perils and prospects. Applied Psychological Measurement, 31(2), 135–157.
McDonald, R.P. (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum.
Zinbarg, R.E., Yovel, I., Revelle, W., and McDonald, R.P (2006). Estimating Generalizability to a Latent Variable Common to All of a Scale’s Indicators: A Comparison of Estimators for $\omega_h$. Applied Psychological Measurement, 30(2), 121–144.

Related Solutions

Solved – Assessing reliability of a questionnaire: dimensionality, problematic items, and whether to use alpha, lambda6 or some other index

I think @Jeromy already said the essential so I shall concentrate on measures of reliability.

The Cronbach's alpha is a sample-dependent index used to ascertain a lower-bound of the reliability of an instrument. It is no more than an indicator of variance shared by all items considered in the computation of a scale score. Therefore, it should not be confused with an absolute measure of reliability, nor does it apply to a multidimensional instrument as a whole. In effect, the following assumptions are made: (a) no residual correlations, (b) items have identical loadings, and (c) the scale is unidimensional. This means that the sole case where alpha will be essentially the same as reliability is the case of uniformly high factor loadings, no error covariances, and unidimensional instrument (1). As its precision depends on the standard error of items intercorrelations it depends on the spread of item correlations, which means that alpha will reflect this range of correlations regardless of the source or sources of this particular range (e.g., measurement error or multidimensionality). This point is largely discussed in (2). It is worth noting that when alpha is 0.70, a widely refered reliability threshold for group comparison purpose (3,4), the standard error of measurement will be over half (0.55) a standard deviation. Moreover, Cronbach alpha is a measure of internal consistency, it is not a measure of unidimensionality and can’t be used to infer unidimensionality (5). Finally, we can quote L.J. Cronbach himself,

Coefficients are a crude device that does not bring to the surface many subtleties implied by variance components. In particular, the interpretations being made in current assessments are best evaluated through use of a standard error of measurement. --- Cronbach & Shavelson, (6)

There are many other pitfalls that were largely discussed in several papers in the last 10 years (e.g., 7-10).

Guttman (1945) proposed a series of 6 so-called lambda indices to assess a similar lower bound for reliability, and Guttman's $\lambda_3$ lowest bound is strictly equivalent to Cronbach's alpha. If instead of estimating the true variance of each item as the average covariance between items we consider the amount of variance in each item that can be accounted for by the linear regression of all other items (aka, the squared multiple correlation), we get the $\lambda_6$ estimate, which might be computed for multi-scale instrument as well. More details can be found in William Revelle's forthcoming textbook, An introduction to psychometric theory with applications in R (chapter 7). (He is also the author of the psych R package.) You might be interested in reading section 7.2.5 and 7.3, in particular, as it gives an overview of alternative measures, like McDonald's $ \omega_t$ or $\omega_h$ (instead of using the squared multiple correlation, we use item uniqueness as determined from an FA model) or Revelle's $\beta$ (replace FA with hierarchical cluster analysis, for a more general discussion see (12,13)), and provide simulation-based comparison of all indices.

References

Raykov, T. (1997). Scale reliability, Cronbach’s coefficient alpha, and violations of essential tau-equivalence for fixed congeneric components. Multivariate Behavioral Research, 32, 329-354.
Cortina, J.M. (1993). What Is Coefficient Alpha? An Examination of Theory and Applications. Journal of Applied Psychology, 78(1), 98-104.
Nunnally, J.C. and Bernstein, I.H. (1994). Psychometric Theory. McGraw-Hill Series in Psychology, Third edition.
De Vaus, D. (2002). Analyzing social science data. London: Sage Publications.
Danes, J.E. and Mann, O.K.. (1984). Unidimensional measurement and structural equation models with latent variables. Journal of Business Research, 12, 337-352.
Cronbach, L.J. and Shavelson, R.J. (2004). My current thoughts on coefficient alpha and successorprocedures. Educational and Psychological Measurement, 64(3), 391-418.
Schmitt, N. (1996). Uses and Abuses of Coefficient Alpha. Psychological Assessment, 8(4), 350-353.
Iacobucci, D. and Duhachek, A. (2003). Advancing Alpha: Measuring Reliability With Confidence. Journal of Consumer Psychology, 13(4), 478-487.
Shevlin, M., Miles, J.N.V., Davies, M.N.O., and Walker, S. (2000). Coefficient alpha: a useful indicator of reliability? Personality and Individual Differences, 28, 229-237.
Fong, D.Y.T., Ho, S.Y., and Lam, T.H. (2010). Evaluation of internal reliability in the presence of inconsistent responses. Health and Quality of Life Outcomes, 8, 27.
Guttman, L. (1945). A basis for analyzing test-retest reliability. Psychometrika, 10(4), 255-282.
Zinbarg, R.E., Revelle, W., Yovel, I., and Li, W. (2005). Cronbach's $\alpha$, Revelle's $\beta$, and McDonald's $\omega_h$: Their relations with each other and two alternative conceptualizations of reliability. Psychometrika, 70(1), 123-133.
Revelle, W. and Zinbarg, R.E. (2009) Coefficients alpha, beta, omega and the glb: comments on Sijtsma. Psychometrika, 74(1), 145-154

Solved – Is it plausible to get a Cronbach’s alpha of .85 with only two Likert-type items

I'm a bit confused by your question. Maybe I miss something in it? Alpha of .85 isn't too high. They say test designed for individual diagnostics / decision making (such as clinical psychological test on result of which important decision about patient are made) alpha should be .90 or higher. The data below shows alpha .855 (and standardized alpha, that is, for standardazed variables, .858). The data look quite realistic, isn't it?

As you know, alpha reflects the magnitude of mean inter-item covariance and variance, and standardized alpha - the magnitude of mean inter-item correlation. When there are only 2 items designed to correlate, it is plausible to expect correlation between them to be higher than the mean correlation between, say, 20 items designed to correlate. For the data above, r is .752 which gives st. alpha $\frac{2*.752}{1+(2-1)*.752}=.858$.

Best Answer

Related Solutions

Solved – Assessing reliability of a questionnaire: dimensionality, problematic items, and whether to use alpha, lambda6 or some other index

References

Solved – Is it plausible to get a Cronbach’s alpha of .85 with only two Likert-type items

Related Question