COVID-19 Economic Impact – Analysis Using Difference-In-Difference Setting for Different Groups

difference-in-differencepanel data

This post provides a clear explanation of a difference-in-difference setting where two similar groups (one with a certain attribute and the other without) are differently affected by the COVID-19 crisis, by comparing them 'before' and 'after' the crisis.

My questions:

1) What are the disadvantages of using this alternative difference-in-difference setting?
2) Are there any (econometric) papers that go into detail about this alternative difference-in-difference setting?

Best Answer

  1. What are the disadvantages of using this alternative difference-in-difference setting?

This alternative approach investigates whether two groups respond differently pre- versus post-event. It assumes a macro-level shock affects all units $i$ in the exact same way. In post you cited, all $i$ (e.g., firms, industries, etc.) are affected by the shock, but one group in particular (i.e., the treatment group), which has a particular characteristic that distinguishes them from the control group, will have a differential response in post-treatment period.

This alternative approach can invite some skepticism. In my opinion, you have to be very clear about how you define your treatment group. You have to disabuse others of the notion that your treatment group was selected "because" it was going to respond differently anyway. If, for example, a macro-shock affects everyone, then you have to demonstrate, in some way, that the outcome trends for the group without the particular trait/characteristic (i.e., the control group) is representative of how the treatment group trend would have evolved had no macro-shock occurred.

In a typical difference-in-differences application, the evaluator is usually not in control of the selection process. If we're lucky, nature does the entire randomization process for us, though never perfectly. In the alternative setting you're referring to, the whole sample is introduced to a new normal. The difference now is that some particular trait or attribute makes one group more/less vulnerable in this new state of the world. The evaluator is usually the one looking for this "feature" that defines treatment status.

Let's look at a real world example.

  1. Are there any (econometric) papers that go into detail about this alternative difference-in-difference setting?

Of course.

A finance paper by Albuquerque and colleagues (2020) is probably the best example, to date, of this alternative approach. They used the COVID-19 pandemic to study how environmental and social (ES) firm policies conditioned the stock market response of firms. In particular, they were interested in testing how customer and investor loyalty based theories of ES account for the stock price properties. Firms with high ES scores (i.e., top quartile in 2018) were considered part of the treatment group. They showed that stock prices for firms with high ES scores perform much better than the prices for other firms. One critique is that some firms/businesses, such as in the utilities industry, were considered "essential" and simply operated in a "business-as-usual" manner in the early stages of the pandemic. If this was the case, this would have resulted in a more resilient business. Maybe there were important differences by industry that were ignored. To address any criticism that the results were driven by a particular industry, they estimated a third difference to show that their findings encompass most industries. Obviously, the distribution of your "treatment group" across sector, industry, or space matters in this setting.

I would give the aforementioned paper a read. The link should take you to the journal where the paper is available (ungated). It outlines this alterative difference-in-differences approach in great detail.

Related Question