I want to perform propensity score matching of observational data of an Intensive Care Unit in order to find out wheather hydroxyethyl starch is better or worse than colloids in terms of renal replacement therapy (RRT), Akute Kidney Injury (AKI) and mortality.
I use the MatchIt package in R (King et al. 2007 – http://gking.harvard.edu/matchit). This package is quite well documented. But there are some things that I dont understand.
First I matched on sociodemographic covariates (as this seems standard protocol with matching): Gender, weight, height and age.
Nearest neighbor matching seems to have worked:
NN matching
m.out.nn
Call:
matchit(formula = treat1 ~ Geschlecht + Gewicht.kg + Groesse.cm +
Alter.bei.ITS.Aufn + BMI, data = hes.vs.kristall.clean, method = "nearest")
Sample sizes:
Control Treated
All 359 3944
Matched 359 359
Unmatched 0 3585
Discarded 0 0
Treatment status treat1 is HES = yes , Colloids otherwise btw. I did a numerical balance check and balance actually WORSENED after matching. Overall as well as some of the covariates drastically:
Call:
matchit(formula = treat1 ~ Geschlecht + Gewicht.kg + Groesse.cm +
Alter.bei.ITS.Aufn + BMI, data = hes.vs.kristall.clean, method = "nearest")
Summary of balance for all data:
Means Treated Means Control SD Control Mean Diff eQQ Med eQQ Mean eQQ Max
distance 0.9168 0.9145 0.0131 0.0022 0.0018 0.0023 0.0388
Geschlecht 0.0041 0.0000 0.0000 0.0041 0.0000 0.0056 1.0000
Geschlechtm 0.6463 0.6100 0.4884 0.0363 0.0000 0.0362 1.0000
Geschlechtw 0.3496 0.3900 0.4884 -0.0403 0.0000 0.0390 1.0000
Gewicht.kg 79.1349 77.8930 18.2092 1.2419 2.0000 1.8462 30.0000
Groesse.cm 169.9184 169.9861 11.9693 -0.0677 0.0000 0.7604 30.0000
Alter.bei.ITS.Aufn 64.5950 63.4808 14.4918 1.1142 0.8000 1.2916 6.2000
BMI 28.4858 27.8005 15.1559 0.6853 0.7080 2.1550 347.8520
Summary of balance for matched data:
Means Treated Means Control SD Control Mean Diff eQQ Med eQQ Mean eQQ Max
distance 0.9357 0.9145 0.0131 0.0212 0.0172 0.0212 0.0687
Geschlecht 0.0446 0.0000 0.0000 0.0446 0.0000 0.0446 1.0000
Geschlechtm 0.9053 0.6100 0.4884 0.2953 0.0000 0.2953 1.0000
Geschlechtw 0.0501 0.3900 0.4884 -0.3398 0.0000 0.3398 1.0000
Gewicht.kg 98.1744 77.8930 18.2092 20.2813 17.0000 20.2813 62.0000
Groesse.cm 164.7103 169.9861 11.9693 -5.2758 2.0000 5.4540 77.0000
Alter.bei.ITS.Aufn 72.5702 63.4808 14.4918 9.0894 7.0000 9.0894 26.5000
BMI 44.1753 27.8005 15.1559 16.3748 6.7500 16.3748 258.9020
Percent Balance Improvement:
Mean Diff. eQQ Med eQQ Mean eQQ Max
distance -852.5593 -839.8420 -808.2029 -77.0723
Geschlecht -998.6072 0.0000 -700.0000 0.0000
Geschlechtm -714.0668 0.0000 -715.3846 0.0000
Geschlechtw -742.6908 0.0000 -771.4286 0.0000
Gewicht.kg -1533.1522 -750.0000 -998.5214 -106.6667
Groesse.cm -7691.0845 -Inf -617.2161 -156.6667
Alter.bei.ITS.Aufn -715.7611 -775.0000 -603.7093 -327.4194
BMI -2289.4307 -853.3898 -659.8482 25.5712
Sample sizes:
Control Treated
All 359 3944
Matched 359 359
Unmatched 0 3585
Discarded 0 0
How can this be possible?
I also did genetic matching (Sekhon 2011 – http://sekhon.berkeley.edu/matching/). This is a fancy algorithm that automatically optimizes covariate balance. There covariate balance has indeed improved (as it should have):
Genetic matching
load(file="m.out.genetic.RData")
Numerical Balance Check
summary(m.out.genetic)
Call:
matchit(formula = treat1 ~ Geschlecht + Gewicht.kg + Groesse.cm +
Alter.bei.ITS.Aufn, data = hes.vs.kristall.clean, method = "genetic")
Summary of balance for all data:
Means Treated Means Control SD Control Mean Diff eQQ Med eQQ Mean eQQ Max
distance 0.9167 0.9147 0.0126 0.0021 0.0019 0.0022 0.0374
Geschlecht 0.0041 0.0000 0.0000 0.0041 0.0000 0.0056 1.0000
Geschlechtm 0.6463 0.6100 0.4884 0.0363 0.0000 0.0362 1.0000
Geschlechtw 0.3496 0.3900 0.4884 -0.0403 0.0000 0.0390 1.0000
Gewicht.kg 79.1349 77.8930 18.2092 1.2419 2.0000 1.8462 30.0000
Groesse.cm 169.9184 169.9861 11.9693 -0.0677 0.0000 0.7604 30.0000
Alter.bei.ITS.Aufn 64.5950 63.4808 14.4918 1.1142 0.8000 1.2916 6.2000
Summary of balance for matched data:
Means Treated Means Control SD Control Mean Diff eQQ Med eQQ Mean eQQ Max
distance 0.9167 0.9164 0.0105 0.0003 0.0018 0.0021 0.0374
Geschlecht 0.0041 0.0000 0.0000 0.0041 0.0000 0.0056 1.0000
Geschlechtm 0.6463 0.6481 0.4782 -0.0018 0.0000 0.0364 1.0000
Geschlechtw 0.3496 0.3519 0.4782 -0.0023 0.0000 0.0392 1.0000
Gewicht.kg 79.1349 79.0556 15.9832 0.0793 2.0000 1.7801 30.0000
Groesse.cm 169.9184 170.0479 10.6992 -0.1296 0.0000 0.7703 30.0000
Alter.bei.ITS.Aufn 64.5950 64.7378 13.3160 -0.1428 0.8000 1.2440 6.2000
Percent Balance Improvement:
Mean Diff. eQQ Med eQQ Mean eQQ Max
distance 83.7418 3.8423 2.7923 0
Geschlecht 0.0000 0.0000 -0.5602 0
Geschlechtm 95.1066 0.0000 -0.5602 0
Geschlechtw 94.3414 0.0000 -0.5602 0
Gewicht.kg 93.6115 0.0000 3.5817 0
Groesse.cm -91.3359 0.0000 -1.2969 0
Alter.bei.ITS.Aufn 87.1817 0.0000 3.6903 0
Sample sizes:
Control Treated
All 359 3944
Matched 357 3944
Unmatched 2 0
Discarded 0 0
I also checked balance graphically and it did improve (despite being good pre-matching).
Now my questions are:
-
Can I use the nearest neighbor matched data? How could I change this so that balance does improve? What kind of distance metric does Nearest neighbor matching use (by default) (Euclidean ?). Because with Euclidean the non-Boolean covariates (Gender) could be made more important than they are.
-
How can I perform analysis after matching? – How can I get the Average Treatment Effect (ATE) and the Average Treatment Effect on the Treated (ATET) in terms of HES for AKI, RRT and mortaility and does that make sense for these response variables (AKI, RRT and mortaility)? Or should I get the odds ratio for Akute Kidney Injury, renal replacement therapy and mortaility from the matched observational data? How do I get these values?
I know that MatchIt recommends using Zelig to get these values but that didn't seem to work with my data.
Can I use logistic regression with the matched data to get the odds ratio of HES vs. Cristalloids of AKI, RRT and mortality ?
Best Answer
1) Using the defaults for MatchIt, nearest neighbor matching matches on the propensity score as defined by a logistic regression of treatment on the covariates included in your formula. For each treated unit, it finds the one unmatched control with the closest propensity score, and then throws out the unmatched control units. There is no issue with continuous vs. categorical covariates here. Note King & Nielsen (2016), who describe why propensity score matching can actually make balance worse, as in your example.
2) MatchIt creates matches for the ATET, but the Matching package, which also implements genetic matching, allows you to specify that you want the ATE. After matching, you can simply perform the regression analysis you would have had you randomly assigned your units (assuming balance has been achieved).