I am developing a research that involves a model and when estimating it the coefficients were not significant. Because the hypotheses are strong, my advisor suspects that there is some problem in the experiment that involves checking the effect of two treatments on the response variable. As the data is quite unbalanced (the observations of one group are much larger than the other) the advisor suggested re-weighting them through entropy balancing. What is the real gain from this? What can I gain with entropy balancing? I couldn't quite understand the base paper I used (Hainmueller, J., 2012). I understood that he can reweight the covars through restrictions on treatment group times. But the real gain I can not understand.
Thanks in advance!
Best Answer
Entropy balancing is a method of equating two groups of units on a specified set of background variables. Conceptually, it is the same thing as matching or inverse probability weighting; indeed, it's just a version of inverse probability weighting where the weights are estimated in a special way.
In an experiment (with perfect compliance and no dropout), no adjustment for the covariates is necessary to get an unbiased estimate of the treatment effect. However, statistical adjustment can be used to improve precision. Methods that equate two groups can be a useful way to remove any slight differences in the distributions of the background variables between the two groups. Adjusting for the variables using regression is another effective, more common method.
For the analysis of an experiment, there is basically no reason to use entropy balancing when you could use regression instead (unless your outcome model is complicated or you want a marginal rather than conditional effect). It's a fancy tool that makes an analysis look technical without actually providing utility over better-understood methods. I would recommend you not use it in this case unless you are forced to. It sounds like you are fishing for the results you want rather than the results the data gave you, which some consider unethical.
For the analysis of observational studies, however, the story is different. Entropy balancing is an incredibly powerful and useful method that almost uniformly outperforms traditional inverse probability weighting. It too must be used with caution and is not always the solution to one's problems, but it should be high on a researcher's list of tools for the analysis of non-experimental data.