Solved – What if only control variables are significant in a differences-in-differences analysis

causalitycontrolling-for-a-variabledifference-in-differenceinterpretationregression coefficients

Regarding the standard DID model:
$$
y=\alpha+\beta_1\text{treat}+\beta_2\text{post}+\beta_3\text{treatâ‹…post}+u
$$
What exactly does it mean if say $\beta_3$ is not statistically significant, but $\beta_1$ is? Does the significance of $\beta_1$ just mean that my control group and my experimental group differed in the very beginning in a statistically significant manner?

In addition, say that you add in additional control variables. Does it matter if the coefficients on any of those control variables is significant, or do we only care about the control variables in how they affect the value and significance of $\beta_3$?

Essentially: how do you interpret the coefficients on your control variables if they end up being significant in a DID regression?

Best Answer

Graphically, this means that you cannot reject the null that your two groups can be represented with two distinct parallel lines. In the graph below, all the $\beta$s are negative. If $\beta_1$ is significant, you know that the treatment group had a significantly lower outcome to start with. If $\beta_3$ is not significantly different from zero, you basically can't tell if there's really a discontinuity in the treatment group line. If $\beta_2$ was not significant, the lines might actually be horizontal, so there's no downward trend.

If some of the coefficients on other explanatory variables are significant, that just tells you that they are associated with $Y$. Depending on the details, it might be possible to give that association a causal interpretation.

Graph taken from The Tarzan blog

Diff-In-Diff