Solved – ATT vs ATE in propensity score matching when using DiD estimates

difference-in-differencepropensity-scorestreatment-effect

According to Lee and Little 2017, when using propensity score (PS) methods, weighting on odds will generate the Average Treatment Effect on the Treated (ATT), while using subclassification and weighting by the inverse probability of treatment (IPTW) will result in the effect being measured on the Average Treatment Effect on the entire sample (ATE).

I believe that Difference-in-Difference (DiD) estimation will generate an ATT. My questions are:

  1. Does the above rule hold true when using PS methods with DiD?
  2. What will the effect measure be when weighing by IPTW in the context of DiD? ATT or ATE?

Best Answer

The article is blocked behind a paywall. Nonetheless, I think the major terms and components can be addressed based on your description.

Propensity score weighting does not weight by the "odds" or weight by the "inverse". Propensity score weighting weights observations by the inverse of the probability of receipt of the treatment.

A difference-in-differences is an estimand, not a response variable. The advantages of ANCOVA, modeling the outcome adjusting for baseline values as a covariate, over a change-score approach have been discussed several times on this site. See here for a lively and thorough discussion. Even so, the difference between the two approaches is a fixed effect vs. an offset; thus the outcome is always just the response variable; hence the formatting of the response variable and interpretation of the treatment receipt coefficient as a difference-in-differences is the same in both approaches.

The average treatment effect on the treated and the average treatment effect (on the sample) is not a designation I've heard before. By definition we estimate the ATE by subtracting a comparable set of differences that would be found in an untreated group. In a clinical study this would be called Hawthorne effect, in observational studies this is usually a type of prevalent case bias. Together, they are types of pre/post differences that do not arise as a form of confounding, so it is not addressable by propensity score weighting.

Conversely, regardless of the presence of these effects, confounding by indication is capable of exaggerating (or attenuating) treatment effects. Propensity score methods (matching or weighting) are still needed to control for confounding effects.