In short, my question is:
Can group-level comparisons (such as t-test, Mann-Whitney tests, etc…) can be used after creating a pseudo-population using inverse probability weighting? (using propensity scores)
Longer version:
Let's assume I am interested in comparing (e.g. t-tests) a control group with patients group, with respect to some continuous outcome, that I know also depends on a continuous covariate (e.g. weight/age/something similar).
After becoming desperate from trying to adjust for the covariate using linear regression, I thought of following a different path and just apply the group-level approaches on the pseudo population I obtain after IPW.
The problem is that I couldn't find any paper following this approach. It seems like the use of IPW is limited to estimating average treatment effects, and not performing group level comparisons. Is there a reason for not using group level comparisons on the pseudo population obtained after IPW?
Thank you
Best Answer
Group comparisons are average treatment effects. They generalize to the population from which the group was sampled, or equivalently, a population that resembles your sample (i.e., in terms of distributions of background characteristics). IPW is an alternative to regression adjustment for controlling for confounding by measured confounders like you have here. When you are comparing the IPW-weighted groups, you are estimating a treatment effect that generalizes to the same population that the unadjusted estimate does, except the estimate is free from confounding (by measured confounders [if done right]).
So, you can perform a "weighted t-test", which compares the weighted groups and estimates an average treatment effect. This weighted t-test compares the weighted means of the outcome in each treatment group. The cool thing about IPW is that the weighted mean of the outcome in the treated group is meant to represent the mean of the outcome in the full sample had everyone received treatment, and the weighted outcome in the control group is meant to represent the mean of the outcome in the full sample had everyone received control. You're still just comparing weighted group means, but the interpretation of these weighted means has a new causal flavor, representing the counterfactual means under each treatment for the whole population.
Rather than doing a weighted t-test, we usually run a weighted least squares regression of the outcome on the treatment, using the IPW weights as the weights and using a robust standard error. The coefficient on treatment in this model is equal to the treatment effect, the difference in weighted means. See here for a guide on doing IPW in R.