Solved – Is the average treatment effect on the treated (ATT) a meaningful comparison in propensity score analysis

propensity-scores

Reading Guo and Fraser's textbook, Propensity Score Analysis: Statistical Methods and Applications (2nd ed.), I came across the formula for the average treatment effect on the treated (ATT) on page 49:

$ATT=E[(Y_1-Y_0)|X,W=1].$

I interpret the ATT formula as the expected treatment outcome for the treated members ($Y_1$) minus the treatment outcome for the untreated members ($Y_0$) given covariates $X$ and given the member received the treatment ($W=1$).

Isn't this a contradictory formula? Perhaps the ATT is meant to measure the difference in the pre-treatment vs. post-treatment outcome ($Y_1-Y_0$) by member given $X$ in a longitudinal dataset? Doesn't this amount to a simple multiple regression?

Best Answer

Saying the propensity score matching estimated quantity is a counterfactual does not make it counterfactual.

According to Pearl, ATT is a counterfactual estimand. This means you cannot find it without knowing ATE (the causal effect) first. Pearl's way to find a counterfactuals is by finding ATE in a SCM setting, replacing the treatment with the alternative treatment (e.g. control) in the equations, and recalculating the outcome from the equations to find the counterfactual effect; i.e., you find the residuals from the causal estimation and use them in the counterfactual.

I think predicting ATT directly from the data using propensity score methods amounts to arbitrarily choosing controls that are not identical to the treatment units; they are just similar, and we do not know how similar. They are similar in the confounding causes, but not similar in the other and unknown causes of the outcome.

A safer method is to find ATE using propensity matching and then use it to find ATT in an independent step using SCM. To me this is more logical. See Pearl's Primer for more details.