I'm a bit confused by your question. Maybe I miss something in it? Alpha of .85 isn't too high. They say test designed for individual diagnostics / decision making (such as clinical psychological test on result of which important decision about patient are made) alpha should be .90 or higher. The data below shows alpha .855 (and standardized alpha, that is, for standardazed variables, .858). The data look quite realistic, isn't it?
1 1
2 1
2 3
3 3
3 3
4 2
4 3
5 7
5 4
5 7
5 6
5 4
5 4
6 6
6 6
6 5
6 3
6 5
6 5
7 6
As you know, alpha reflects the magnitude of mean inter-item covariance and variance, and standardized alpha - the magnitude of mean inter-item correlation. When there are only 2 items designed to correlate, it is plausible to expect correlation between them to be higher than the mean correlation between, say, 20 items designed to correlate. For the data above, r is .752 which gives st. alpha $\frac{2*.752}{1+(2-1)*.752}=.858$.
If data are MCAR, one would like to find an unbiased estimated of alpha. This could possibly be done via multiple imputation or listwise deletion. However, the latter might lead to severe loss of data. A third way is something like pairwise deletion which is implemented via an na.rm
option in alpha()
of the ltm
package and in cronbach.alpha()
of the psych
package.
At least IMHO, the former estimate of unstandardized alpha with missing data is biased (see below). This is due to the calculation of the total variance $\sigma^2_x$ via var(rowSums(dat, na.rm = TRUE))
. If the data are centered around 0, positive and negative values cancel each other out in the calculation of rowSums
. With missing data, this leads to a bias of rowSums
towards 0 and therefore to an underestimation of $\sigma^2_x$ (and alpha, in turn). Contrarily, if the data are mostly positive (or negative), missings will lead to a bias of rowSums
towards zero this time resulting in an overestimation of $\sigma^2_x$ (and alpha, in turn).
require("MASS"); require("ltm"); require("psych")
n <- 10000
it <- 20
V <- matrix(.4, ncol = it, nrow = it)
diag(V) <- 1
dat <- mvrnorm(n, rep(0, it), V) # mean of 0!!!
p <- c(0, .1, .2, .3)
names(p) <- paste("% miss=", p, sep="")
cols <- c("alpha.ltm", "var.tot.ltm", "alpha.psych", "var.tot.psych")
names(cols) <- cols
res <- matrix(nrow = length(p), ncol = length(cols), dimnames = list(names(p), names(cols)))
for(i in 1:length(p)){
m1 <- matrix(rbinom(n * it, 1, p[i]), nrow = n, ncol = it)
dat1 <- dat
dat1[m1 == 1] <- NA
res[i, 1] <- cronbach.alpha(dat1, standardized = FALSE, na.rm = TRUE)$alpha
res[i, 2] <- var(rowSums(dat1, na.rm = TRUE))
res[i, 3] <- alpha(as.data.frame(dat1), na.rm = TRUE)$total[[1]]
res[i, 4] <- sum(cov(dat1, use = "pairwise"))
}
round(res, 2)
## alpha.ltm var.tot.ltm alpha.psych var.tot.psych
## % miss=0 0.93 168.35 0.93 168.35
## % miss=0.1 0.90 138.21 0.93 168.32
## % miss=0.2 0.86 110.34 0.93 167.88
## % miss=0.3 0.81 86.26 0.93 167.41
dat <- mvrnorm(n, rep(10, it), V) # this time, mean of 10!!!
for(i in 1:length(p)){
m1 <- matrix(rbinom(n * it, 1, p[i]), nrow = n, ncol = it)
dat1 <- dat
dat1[m1 == 1] <- NA
res[i, 1] <- cronbach.alpha(dat1, standardized = FALSE, na.rm = TRUE)$alpha
res[i, 2] <- var(rowSums(dat1, na.rm = TRUE))
res[i, 3] <- alpha(as.data.frame(dat1), na.rm = TRUE)$total[[1]]
res[i, 4] <- sum(cov(dat1, use = "pairwise"))
}
round(res, 2)
## alpha.ltm var.tot.ltm alpha.psych var.tot.psych
## % miss=0 0.93 168.31 0.93 168.31
## % miss=0.1 0.99 316.27 0.93 168.60
## % miss=0.2 1.00 430.78 0.93 167.61
## % miss=0.3 1.01 511.30 0.93 167.43
Best Answer
Cronbach's $\alpha$ is only designed for measures that are essentially $\tau$-equivalent, which essentially means they contribute equally to the underlying construct. One way to test this is to see if they have the same factor loadings in a factor model. If your measures are not $\tau$-equivalent, then $\alpha$ will underestimate reliability, regardless if the data are continuous or dichotomous.
There are many indices of internal-consistency reliability. $\omega$ (omega) as identified by McDonald (1999) is one of the most flexible for unidimensional constructs, and it can be easily extended to a multidimensional construct. Here is a procedure I recommend taking to identify which measure of reliability to use:
1) First assess dimensionality. Do you have 1 construct or many? If there are many, then no measure of unidimensional reliability will be accurate. Do this with factor analysis, ideally confirmatory factor analysis (CFA), but if you don't have the knowledge or the software you can use exploratory factor analysis (EFA). If you have more than 1 factor that is substantive, then you have a multidimensional construct. If that's the case, look for a measure of multidimensional reliability (these exist for both $\alpha$ and $\omega$. See here. Alternatively, identify the items that don't fit your desired construct and remove them (though take caution here, there are a lot of other psychometric tests you should do as well).
2) Assess $\tau$-equivalence. Again doing this in a factor model may be easiest. Basically, you test to see if the loadings are all equal - in a CFA, you can constrain the loadings and test fit, in an EFA you just have to ballpark the loadings to see if they are reasonably close. If you have $\tau$ equivalence, go ahead and use $\alpha$. If not, use $\omega$.
From what I can tell, SPSS does not calculate $\omega$ (see here). In my view, R is one of the best packages out there for psychometrics because it has the flexibility to do all of this. If you don't know R and don't have the time/energy to learn it (it's a big leap from SPSS) then you can probably safely go with $\alpha$ if you construct is unidimensional, just keep in mind reliability will be higher than what $\alpha$ gives you.
Reference: McDonald, R. P. (1999). Test Theory: A Unified Treatment. New York: Psychology Press.