Solved – How to run chi-squared test on imputed data

chi-squared-testdata-imputation

I have a survey data set with missing values and I generated 10 multiple imputations in which the missing values were imputed. There are several categorical variables in the data sets and I'd like to see if there's any association among these variables using chi-squared test.

However, I've been searching how to run this analysis in either SAS or Stata but didn't find a solution. It seems that mi estimate doesn't support svy: tab in Stata. I can run chi-squared test on each imputed data set using in SAS using proc surveyfreq, however, I don't know how to combine the results. Can anyone please guide me through the steps on how to test the associations between categorical variables?

Thank you in advance!
Jin

Best Answer

How to do this has been elaborated by Rubin (1988, p.87) and Li et al. (1991).

First, you take mean $\chi^2$ across multiply imputed data sets.

$$\chi^2=\frac{1}{m}\sum _{l=1}^{m}\chi^2_l$$

You estimate the relative increase in variance:

$$r=(1+\frac{1}{m})\frac{1}{m-1}\sum _{l=1}^{m}(\sqrt{\chi^2_l}-\sqrt{\chi^2})^2$$

And the test statistic:

$$D_x=\frac{\frac{\chi^2}{k}-\frac{m+1}{m-1}r}{1+r}$$

where $k$ is the degrees of freedom of $\chi^2$. Now, $D_x$ is F-distributed:

$$P_x = Pr(F_{k,v} \gt D_x)$$

where

$$v=k^{-3/m}(m-1)(1+\frac{1}{r})^2$$

which gives you the Chi-square test across multiply imputed data sets.

Li, K.-H., Meng, X.-L., Raghunathan, T. E., & Rubin, D. B. (1991). Significance levels from repeated p-values with multiply-imputed data. Statistica Sinica, 1(1), 65–92.

Rubin, D. B. (1988). Multiple Imputation for Nonresponse in Surveys. John Wiley & Sons.

Related Solutions

Solved – How to handle missing data when determining differences between groups using chi-squared or Fisher’s exact test

Unless there is some specific reason for people being NA, and unless you are interested in that reason, then I would say to not include people who are missing.

You don't need an exact test here; all the cell sizes are reasonable.

However 1) Don't you want some form of regression instead? and 2) Why is Albumin dichotomized into low and high? Dichotomizing continuous variables is usually a bad idea (see Royston, Altman & Sauerbrei).

If you have actual values for albumin, I suggest a linear regression albumin~case, possibly with other covariates added, if you have data. This is especially important if this is an observational study, but is still worthwhile if it is an experimental one, because covariates can vary between groups, even if assignment is random, and because covariates can affect other regressors.

Solved – Chi squared test result and Cramer’s V value

Because your sample size is large, the Chi-square test is likely to return a low p-value even for a table with small differences from the expected proportions.

To get a sense of the effect size being reported by Cramer's v, it is helpful to look at the proportions in the table. For example, for column 1, you can see that there is not much difference in the proportions for each grade within the rows. Row 0 is about one-half to 1 percent of the observations in each grade. Row 2 is, say, 93 to 97 percent of observations in each grade. And so on.

Whether these kind of differences in proportions are meaningful in your context is up to you. The p-value and Cramer's v give you certain information. The practical importance of your results is something you will have decide.

The following is code for R.

I am getting a slightly different Cramer's v than you, so that's something that you might want to look into.

Input =("
Col1               Grade1  Grade2   Grade3   Grade4   Grade5
0                   290      392      932     1812     2854
1                   522      421      574      917     1247
2                 56789    81296   117971   147811   204480
3                  3719     2975     2811     1704     2244
")

Matrix = as.matrix(read.table(textConnection(Input),
                   header=TRUE,
                   row.names=1))

Matrix

chisq.test(Matrix)

   ### Pearson's Chi-squared test
   ###
   ### X-squared =  8113.9, df = 12, p-value < 2.2e-16

library(vcd)

assocstats(Matrix)

   ### Cramer's V        : 0.065 

prop.table(Matrix, margin=2)

   ###        Grade1      Grade2      Grade3      Grade4      Grade5
   ### 0 0.004729289 0.004607212 0.007621353 0.011901947 0.013537294
   ### 1 0.008512720 0.004948051 0.004693837 0.006023226 0.005914858
   ### 2 0.926108937 0.955479291 0.964698090 0.970882268 0.969903949
   ### 3 0.060649054 0.034965446 0.022986720 0.011192559 0.010643899

Best Answer

Related Solutions

Solved – How to handle missing data when determining differences between groups using chi-squared or Fisher’s exact test

Solved – Chi squared test result and Cramer’s V value

Related Question