Solved – Difference-in-difference: fixed effect vs. clustered standard error

Question

I am trying to run a difference-in-difference regression. I have one country in the treatment group and two countries in the control group. I believe there is a need to account for fixed effects to account for the unobserved endogeneity across countries but, should I also include clustered standard errors to account for the unobserved heterogeneity within groups?

I have tried running this in Stata. Both regressions produce an interaction term which is significant when I do the regressions individually, however when I run the regression with clustered standard errors and account for fixed effects, the treatment term becomes massively insignificant. Why is this?

Answer 1

For the purposes of this answer, I will assume your data has a typical panel structure.

I believe there is a need to account for fixed effects to account for the unobserved endogeneity across group but, should I also include clustered standard errors to account for the unobserved heterogeneity within groups?

The difference-in-differences (DD) formulation includes fixed effects for cross-sectional units. Because of this, DD allows for some selection into treatment on the basis of unobserved, time-invariant characteristics. It is worth highlighting, however, that allowing for country-specific intercepts does not account for the within-unit (country) dependence among your observations.

Practitioners typically apply cluster-robust uncertainty estimators to account for the within-group residual correlation. Based upon how your data is structured, you should suspect correlation in the measurements within-clusters. In your particular setting, all of your 'country-year' observations are not independent pieces of new information; your observations are likely correlated across time within a country.

Both regressions produce an interaction term which is significant when I do the regressions individually, however when I run the regression with clustered standard errors and account for fixed effects, the treatment term becomes massively insignificant. Why is this?

In settings where the number of clusters is small, clustering your standard errors at the 'country level' may not work well. As indicated in Cameron and Miller's (2015) practitioner's guide to cluster-robust variance estimation/inference, it is not uncommon to observe your "default" (i.e., unadjusted) standard errors to be several times smaller than those that account for the dependence among observations. Please review their guide as they demonstrate many useful correction methods and a diverse range of practical applications in Stata.

In general, practitioners/scholars argue that cluster-robust inference is compromised when the number of clusters falls below 50. I would argue, however, that there is no clear consensus in any literature thus far regarding this 'cut-off' for clusters. You should peruse the following post for more information on this topic.