Scribd
Upload
66 views15 pages

Causal Inference

The document summarizes key concepts in impact evaluation presented by Dr. Sarah Dong at the IRSA Annual Conference workshop in Solo, Indonesia in 2018. It discusses the counterfactual approach to estimating program effects, and challenges such as the fundamental problem that the outcome for each individual under both treatment and no treatment can never be observed. It then covers different methods for approximating the counterfactual, such as matching, regression discontinuity design, differences-in-differences, and fixed effects. The document also defines average treatment effects, average treatment effects on the treated, and local average treatment effects, and discusses how these estimates may differ depending on how representative the sample is of the target population.

Uploaded by

Afif
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
66 views15 pages

Causal Inference

The document summarizes key concepts in impact evaluation presented by Dr. Sarah Dong at the IRSA Annual Conference workshop in Solo, Indonesia in 2018. It discusses the counterfactual approach to estimating program effects, and challenges such as the fundamental problem that the outcome for each individual under both treatment and no treatment can never be observed. It then covers different methods for approximating the counterfactual, such as matching, regression discontinuity design, differences-in-differences, and fixed effects. The document also defines average treatment effects, average treatment effects on the treated, and local average treatment effects, and discusses how these estimates may differ depending on how representative the sample is of the target population.

Uploaded by

Afif
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 15

Impact Evaluation Workshop

IRSA Annual Conference, Solo, 2018

Causal Inference

Dr. Sarah Dong


Fellow, Indonesia Project
Australian National University
[email protected]
Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 2
The counterfactual
What would have happened to
beneficiaries had they not received the
intervention?
Outcome
Observed from
Effect = outcome - counterfactual

Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 3


The missing counterfactual
Outcome A: Outcome B:
Effect = receive - do not
PKH receive PKH

Recipient: A - B

Non-recipient: A - B

but never observe both A and B for the same


household!

Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 4


Problem with OLS regression
Y = αTreat+ε
Treat =1 if receiving program
Treat = 0 if not receiving program

How do we interpret α?
• α is the average difference in Y between people who
receive the program and people who do not receive the
program
• These two groups of people may be different and not
comparable

Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 5


Approximate for the counterfactual?
• Find a household in non-recipients that is
similar to the household in recipients
• The similar household will be the
counterfactual

Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 6


What is a similar household in the non-
recipient pool?
• Similar in observed characteristics
– OLS, Matching
• Similar in both observed and unobserved
characteristics
– RCT, RDD, IV
• Similar in change overtime
– DID
• Same household in a pre-program time
– Panel fixed effect
Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 7
Exercise: good counterfactual vs. bad
counterfactual vs.

Program: PKH in Papua


Outcome: education of children in the recipient household
Unit: household
• Counterfactual 1: Non-recipient households in the same
village
• Counterfactual 2: Non-recipient household with similar
size, income, household composition in Jakarta
• Counterfactual 3: Education of children in the same
household 5 years ago before introduction of PKH
• Ideal counterfactual?

Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 8


Exercise: good counterfactual vs. bad
counterfactual vs.
Program: Dana desa
Outcome: average per capita expenditure of the village
Unit: village
• Counterfactual 1: Villages that receive a lower amount of
village fund
• Counterfactual 2: Urban areas that do not receive village
fund
• Counterfactual 3: The same village when there was no
village fund
• Ideal counterfactual?

Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 9


Exercise: good counterfactual vs. bad
counterfactual vs.

Program: Tax amnesty


Outcome: tax revenue
Unit: national
• Counterfactual 1: tax revenue in the same months in the
year before the amnesty
• Counterfactual 2: tax revenue in the same months in the
year after the amnesty
• Ideal counterfactual?

Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 10


The treatment effects
• Average treatment effect (ATE)

• Average treatment effect on the treated


(ATT)

• Local average treatment effects (LATE)


(for later: IV and RDD)

Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 11


Average treatment effect
Outcome B1:
Outcome A1: do not
Individual 1: treatment effect1 = receive receive
program program

Individual 2: treatment effect2 = Outcome A2 Outcome B2

Individual N: treatment effectN = Outcome AN Outcome BN

Average treatment effect = average(treatment effect1, …, treatment effectN)


Note: N is the population

Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 12


Average treatment effect on the treated
Individual 1: treatment effect1 = Outcome A1: Outcome B1:

Treated …
Individual N1: treatment effectN1 = Outcome AN1 Outcome BN1

Outcome Outcome
Individual N1+1: treatment effectN1+1 = BN1+1
AN1+1
Not
Treated …
Individual N: treatment effectN = Outcome AN Outcome BN

Average treatment effect on the treated =


Average(treatment effect1, …, treatment effectN1)

Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 13


ATE vs. ATT
• ATE is usually unobtainable, unless
– Sample is representative from population
– Random allocation between treated and
untreated
• ATT is usually what we get
• Therefore should state for what group is
the estimated treatment effect for

Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 14


Exercise: defining treatment effects
For PKH, define:
1. The ATE
2. The ATT
3. The potential difference between ATE and
ATT

Sarah Dong (ANU) IRSA Workshop July 21-22, 2018 15

You might also like