Objective: I want to test the effect of a regulatory change using a classical pre-post/treatment-control DiD design (Y = POST + TREAT + POSTxTREAT + e).
Problem: Treatment & control obs. are from different countries. Although these countries are similar with respect to many relevant aspects, a visual check of the parallel trend assumption for the outcome variable in the pre-period (only 3 years) indicates different time trends.
Sample details: time period is 2002-2007, event happened in 2005. Treatment obs. 200/year (1200 total), control obs. ~ 100 obs./year (600 total). Obs. over time belong to the same subjects (300 subjects).
Questions:
-
Does it make sense to use propensity-score matching based on determinants of the outcome variable in the pre-period to create a more homogenous treatment/control sample (and hopefully get rid of the different trends)?
-
What would be the advantage over simply controlling for the determinants used in the PSM in the DiD regression? (e.g., Y = POST + TREAT + POSTxTREAT + CONTROLS + e)
-
I thought of using one-on-one matching with a caliper. However, I have more treatment obs. than control obs. – are there any better ways to avoid losing too many of my obs.?
Thanks!
PS: I'm using Stata.
Best Answer
Propensity score matching before a difference in differences analysis is a potent way to get around different parallel trends in the pre treatment period and has been used in several papers (e.g. Becker and Hvide, 2013; Ichino et al., 2007). So it definitely does make sense.
The advantage over simply including country dummies in your difference in differences regression is that with the dummies only you will still keep all the observations with different trends. This will not solve the problem. The matched sample on the other hand will only have those observations with the common pre-treatment trend. Nonetheless it is worthwhile to include other control variables because these can help to soak up the residual variance, hence your inference will be more precise. This is particularly true if the treatment was random.
Instead of using the caliper it is probably a good idea to start off with the simpler propensity or nearest neighbor matching algorithms. After the matching check the summary statistics of your outcome and explanatory variables before the treatment for the treatment and control group separately. They should be fairly similar if your matching was successful. You can then test for whether the covariates are balanced and the common support assumption (if this doesn't sound familiar have a look at practitioner's guide to propensity score matching by Caliendo and Kopeinig, 2005). If you are still concerned about the quality of your matches after that, you can still do the caliper matching and see whether the results change dramatically. In the best case they shouldn't change a lot. The issue with caliper matching is that the choice of the caliper is relatively arbitrary and you will always have to defend the choice of a particular radius value.
With respect to the fact that you have more treatment than control observations you can use matching with replacement. This way some of the controls will be used more than once if they fit well with more than one treatment unit.
Given that you are using Stata, perhaps you are interested in the practical implementation of propensity score matching with panel data with Stata. I hope this helps.