Annals of Clinical Epidemiology 2021;3(3):74–77

SEMINAR

Introduction to Difference-in-differences Design

Yusuke Sasabuchi
Data Science Center, Jichi Medical University

ABSTRACT
Because it is difficult to conduct randomized controlled trials, observational studies are often used when evaluating the
effects of health care policies. However, observational studies are subject to bias, such as a failure to eliminate the effects
of trends in the outcome over time and permanent differences between treatment and control groups. The difference-
in-differences design removes these biases by observing outcomes for the two groups at two time points. This article
introduces the methods and assumptions for the difference-in-differences design and provides some examples of studies
that have used this design.

KEY WORDS
Difference-in-differences design, parallel trends, common shocks

groups. For example, lung cancer may be consistently

I NTRO DU CT IO N
more prevalent in areas that have implemented the inter‐
Because it is difficult to conduct randomized controlled vention than in areas that have not, but the cohort study
trials, observational studies are often used when evaluat‐ design is not equipped to eliminate the effects of this per‐
ing the effects of health care policies. In this context, two manent difference.
study designs are considered: the before-and-after design The difference-in-differences design removes the
and the cohort study design. The before-and-after design biases mentioned above by observing outcomes for two
compares a group before the implementation of an inter‐ groups at two time points. This article introduces the
vention with another group after the implementation of methods and assumptions for difference-in-differences
the intervention to estimate the effect of the intervention. design and provides some examples of studies that have
However, in the before-and-after design, it is not possible used this design.
to determine whether the estimated effect is an actual
effect of the intervention or a trend in the outcome over
W H A T I S D I F F E R E N C E -I N - D I F F E R E N C E S
time [1, 2]. For example, when using a before-and-after
ANALYSIS?
design to examine the effects of a policy aimed at pre‐
venting passive smoking, it is not possible to eliminate The difference-in-differences design sets up a control
the effects of a decreasing trend in the number of people group that has the same trend over time as the interven‐
smoking in public places that began before the policy was tion group but that is not affected by the intervention [3].
implemented. The intervention group is exposed to the intervention at
Conversely, cohort studies compare a group in a spe‐ the second time point but not at the first time point,
cific area where an intervention has been implemented whereas the control group is not exposed to the interven‐
with another group in a different area where the inter‐ tion at either time point. The intervention effects are esti‐
vention has not been implemented. However, effects esti‐ mated by subtracting the effects in the control group
mated in cohort studies are often biased toward perma‐ from the effects in the treatment group. In this way, the
nent differences between the treatment and control difference-in-differences design is able to eliminate the

74
 Introduction to difference-in-differences design

influences of trends and permanent differences between we are taking the difference between Difference 2 and
the two groups and correctly evaluate the effect of the Difference 1, this design is called “difference-in-
intervention. differences.”
Fig. 1 shows a conceptual diagram of difference-in-
differences analysis. In the control group, the outcome
B A S I C M E T H O D S F O R D I F F E R E N C E - I N-
changes from C1 to C2; in the intervention group, the
DIFFERENCES ANALYSIS
outcome changes from T1 to T2. The intervention group
has the same trend as the control group, and, if there Regression models are commonly used to estimate the
were no intervention in the intervention group, the out‐ effect of an intervention [2, 3]. Three variables are inclu‐
come in the intervention group at the second time point ded in the basic difference-in-differences model: (i) the
would be T2'. Therefore, the effect of the intervention is group assignment variable (presence or absence of the
T2 − T2'. Difference 1, the change in the outcome before intervention), (ii) time point (before or after the inter‐
and after the intervention in the control group, is vention), and (iii) the interaction term of these two fac‐
expressed as “C2 − C1.” Difference 2, the change in out‐ tors (multiplication of the two factors). The interaction
come for the intervention group over the same period, is term is the difference-in-differences estimator.
expressed as “T2 − T1.” Because the intervention and The equation of the difference-in-differences regres‐
control groups have the same trend, “C2 − C1” and “T2' sion model is
− T1” are equal. Therefore, the effect of the intervention Y = α + β1 * (intervention) + β2 * (time point) + β3 *
is expressed as Difference 2 − Difference 1 = (T2 − T1) − (intervention) * (time point),
(C2 − C1) = (T2 − T1) − (T2' −T1) = T2 − T2'. Because where Y is the outcome of interest, intervention is a
dummy variable for group assignment (intervention
group = 1, control group = 0), and time point is a dummy
variable for the time point (post-intervention = 1, pre-
intervention = 0).
Table 1 shows how to estimate the intervention effect
from the regression equation. The outcome before the
intervention in the intervention group is α + β1, substitut‐
ing 1 for intervention and 0 for time point in the above
regression equation. The post-intervention outcome for
the intervention group is α + β1 + β2 + β3, substituting 1
for intervention and for time point. Therefore, the change
in the outcome of the intervention group is (α + β1 + β2 +
β3) − (α + β1) = β2 + β3. Similarly, the change in the out‐
come of the control group before and after the interven‐
Fig. 1 Conceptual diagram of the difference-in-differences design tion is calculated as (α + β2) − α = β2. The difference
T1and T2 indicate the outcomes in the intervention group before and between these two outcomes, (β2 + β 3) − β 2 = β 3, which is
after the intervention. C1 and C2 indicate the outcomes in the con‐
the coefficient of the interaction term, shows the inter‐
trol group before and after the intervention. If the trends in the inter‐
vention and control groups are parallel, the post-intervention out‐ vention effect.
come in the intervention group would be T2' if this group did not
receive the intervention. Therefore, the effect of the intervention is
estimated as T2 − T2'.

Table 1 Estimation of the effect of the intervention in regression equation.

Before introduction of After introduction of

After–Before
the intervention (time point = 0) the intervention (time point = 1)

The intervention group (intervention = 1) α + β1 α + β1 + β2 + β3 β2 + β3 (=difference 2)

The control group (intervention = 0) α α + β2 β2 (=difference 1)

Intervention-control β1 β1 + β3 β3 (=difference-in-differences)

75
 ANNALS OF C L I N I C A L E P I D E M I O L O G Y

same impact on outcomes in both the intervention group

A SSU MP TI ONS OF THE D I F F E REN CE -IN -
and the control group [1, 4]. If the assumption does not
D I F F ERE NC ES DE SI GN
hold, the effect of the intervention cannot be estimated
1. Parallel Trends correctly using the difference-in-differences design.
The parallel trends assumption is that the change in the Unfortunately, this assumption cannot be verified from
outcome over time is similar in the intervention and con‐ the data. When interpreting research using a difference-
trol groups during the observation period [1–3]. If there in-differences design, attention should be paid to any
are parallel trends in the changes in outcomes for the events occurring during the observation period.
intervention and control groups, the difference-in-
differences design correctly estimates the intervention
RESEARCH EXAMPLES USING
effect. Although we cannot see the change in the outcome
DIFFERENCE-IN-DIFFERENCES ANALYSIS
for the intervention group in the absence of the interven‐
tion, if the trends of the two groups are parallel before the This section presents examples of studies using the
intervention, it makes sense to assume that the trends difference-in-differences design.
will be parallel after the intervention.
To check whether this assumption holds, a comparison 1. “Impact of Centralizing Acute Stroke Services in English
is often made by plotting the trends of the two groups Metropolitan Areas on Mortality and Length of Hospital
before the intervention. Fig. 2 shows schematic diagrams Stay: Difference-in-Differences Analysis” [5]
of trends before the intervention. If the pre-intervention In 2010, acute stroke services were centralized in London
trends are parallel, we can assume that the post- and Greater Manchester in England. This study investiga‐
intervention trends are also parallel. Conversely, if the ted whether the centralization of acute stroke services
pre-intervention trends are not parallel, the difference-in- reduced 90-day mortality and length of hospital stay,
differences design cannot correctly estimate the effect of using a difference-in-differences design.
the intervention. Another option to check the parallel This study used the Hospital Episode Statistics data‐
trends assumption is to test whether the pre-intervention base, which contains inpatient data from treatment cen‐
trends are statistically different between the intervention ters funded by the National Health Service. Centraliza‐
and control groups. tion of acute stroke care occurred in April 2010 in
Greater Manchester and in July 2010 in London. The
2. Common Shocks study participants were inpatients with stroke living in
A shock is an unexpected event that is unrelated to the urban areas from January 2008 to March 2012. The inter‐
intervention. According to the common shocks assump‐ vention group comprised patients with stroke living in
tion, such an event occurring at the same time as or fol‐ London or Greater Manchester, and the control group
lowing the implementation of the intervention has the comprised those living in other areas.

Fig. 2 Parallel trends assumption of the difference-in-differences design

The left panel shows parallel pre-intervention trends. In this case, the effect of the intervention can be correctly estimated. The right panel shows a
violation of the parallel trends assumption. If this assumption is violated, the effect estimated by difference-in-differences analysis is invalid.

76
 Introduction to difference-in-differences design

The regression model in this study was prised patients who underwent total hysterectomy at
Y = α + β1 * (Area) + β2 * (time point) + β3 * (Area) * the same institution. Preoperative administration of pro‐
(time point), phylactic antimicrobial agents for total hysterectomy was
where Y is the outcome of interest, Area is a dummy vari‐ recommended throughout the study period.
able for residence in the two areas where stroke services The regression model in this study was
were centralized (Area = 1 for London or Greater Man‐ SSI = α + β1 * (CS) + β2 * (time point) + β3 * (CS) *
chester, Area = 0 for other areas), time point is a dummy (time point),
variable for the time point (time point = 0 for before the where SSI is surgical site infection, CS is a dummy varia‐
centralization, time point = 1 for after the centralization), ble for cesarean section (CS = 1 for cesarean section, CS
and β3 is a difference-in-differences estimator. = 0 for total hysterectomy), time point is a dummy varia‐
The comparison of pre-intervention mortality and ble for the time point (time point = 0 for prior to January
length of stay between the two centralized areas and 2014, time point = 1 for January 2014 or later), and β3 is a
other areas showed no significant difference, which difference-in-differences estimator.
means the parallel trends assumption was not violated. The results showed no statistically significant differ‐
The results showed that 90-day mortality was signifi‐ ence in the risk of surgical site infection after the change
cantly decreased in London (−1.1%, 95% confidence in policy in January 2014: −0.6% (p = 0.663).
interval: −2.1 to −0.1) but not in Greater Manchester It is unclear whether the two assumptions of the
(0.1%, 95% confidence interval: −1.1 to 1.3). Length of difference-in-differences design were met in this study
hospital stay was significantly decreased in both areas because there is no description of either parallel trends or
(−1.4 days, 95% confidence interval: −2.3 to −0.5 in common shocks.
London; −2.0 days, 95% confidence interval: −2.8 to −1.2
in Greater Manchester).
CONCLUSIONS
2. “Timing of Antibiotic Prophylaxis in Cesarean Section: Before-and-after studies and cohort studies are subject to
Retrospective, Difference-in-Differences Estimation of the bias, such as the inability to identify the effects of trends
Effect on Surgical Site Infection” [6] over time and permanent differences between groups.
This study used a difference-in-differences design to Difference-in-differences analysis eliminates the effect of
determine whether changing the timing of prophylactic trends over time and permanent differences between the
antimicrobial administration for cesarean section from groups. Two critical assumptions in the difference-in-
immediately after cord ligation to preoperative adminis‐ differences design involve parallel trends and common
tration reduced the incidence of surgical site infection. shocks.
This study was a single-center report from the facility
with the highest number of deliveries in Israel, observed CONFLICT OF INTEREST
over four years from January 1, 2012, to December 31, None
2015. Before January 2014, prophylactic antimicrobial
agents were administered immediately after cord ligation ACKNOWLEDGMENTS
for cesarean sections at this facility. In January 2014, the This work was supported by the Ministry of Education, Culture,
policy was changed to administering the agents immedi‐ Sports, Science and Technology (Grant Numbers 19K19394,
ately before the start of surgery. The control group com‐ 21H03159).

REFERENCES

1. Dimick JB, Ryan AM. Methods for evaluat‐ Princeton, NJ: Princeton University Press; ropolitan areas on mortality and length of
ing changes in health care policy: the 2008. hospital stay: difference-in-differences analy‐
difference-in-differences approach. JAMA 4. Ryan AM, Burgess JF Jr, Dimick JB. Why sis. BMJ 2014;349:g4757.
2014;312:2401–2. We Should Not Be Indifferent to Specification 6. Ben Shoham A, Bar-Meir M, Ioscovich A,
2. Abadie A. Semiparametric difference-in- Choices for Difference-in-Differences. Health Samueloff A, Wiener-Well Y. Timing of anti‐
differences estimators. Rev Econ Stud 2005; Serv Res 2015;50:1211–35. biotic prophylaxis in cesarean section: retro‐
72:1–19. 5. Morris S, Hunter RM, Ramsay AI, Boaden spective, difference-in-differences estimation
3. Angrist JD, Pischke JS. Mostly Harmless R, McKevitt C, Perry C, et al. Impact of cen‐ of the effect on surgical-site-infection.
Econometrics: An Empiricist’s Companion. tralising acute stroke services in English met‐ JMFNM 2019;32:804–8.

77