Week 4

Exercise at home: (𝑅𝑒𝑓𝑒𝑟𝑒𝑛𝑐𝑒 𝐸𝑥𝑎𝑚𝑝𝑙𝑒)

𝑿 𝒀
1 0
0 2
-1 0
2 3
• Compute 𝛽̂𝑂𝐿𝑆 using Matrix Form.
0
2
𝑌=( )
0
3
1 1
1 0
𝑋=( )
1 −1
1 2
𝛽̂𝑂𝐿𝑆 = (𝑋 ′ 𝑋)−1 𝑋′𝑌

1 1
1 1 1 1 1 0 4 2
(𝑋 ′ 𝑋 ) = ( ) ( ) =( )
1 0 −1 2 2×4 1 −1 2 6 2×2
1 2 4×2
𝑎 𝑏
𝐴=( )
𝑏 𝑐

1 𝑐 −𝑏
𝐴−1 = ( )
|𝑎𝑐 − 𝑏 2 | −𝑏 𝑎
 1 6 −2
(𝑋 ′ 𝑋)−1 = ( )
20 −2 4 2×2

0
1 1 1 1 2 5
(𝑋 ′ 𝑌) = ( ) ( ) =( )
1 0 −1 2 2×4 0 6 2×1
3 4×1

1 6 −2 5 0.9 𝛼̂
𝛽̂𝑂𝐿𝑆 = ( ) ( ) =( ) =(̂ )
20 −2 4 2×2 6 2×1 0.7 2×1 𝛽1 2×1

𝑌̂ = 0.9 + 0.7𝑋
𝑊ℎ𝑒𝑛 𝑋 𝑖𝑛𝑐𝑟𝑒𝑎𝑠𝑒𝑠 𝑏𝑦 1 𝑈𝑛𝑖𝑡, 𝑡ℎ𝑒 𝑌 𝑖𝑛𝑐𝑟𝑒𝑎𝑠𝑒𝑠 𝑏𝑦 0.7 𝑈𝑛𝑖𝑡𝑠.
So far, we learned
(1) 𝐷𝑒𝑟𝑖𝑣𝑎𝑡𝑖𝑜𝑛 𝛽̂
(2) 𝑁𝑢𝑚𝑒𝑟𝑖𝑐𝑎𝑙 𝐴𝑝𝑝𝑙𝑖𝑐𝑎𝑡𝑖𝑜𝑛 𝛽̂

Once we have 𝛽̂ we are concerned about its significance. 𝛽̂ might have a

value but it is not significant (Its effect is like the Zero)
 𝐻𝑜 : 𝛽̂ = 𝛽 = 0
𝐻1 : 𝛽̂ ≠ 𝛽 ≠ 0
To be able to run the test above, you need information about 𝑉(𝛽̂ ).
Therefore, the goal is to derive 𝑉(𝛽̂ ).
 ̂ ) Derivation:
𝑽(𝜷
Math Notes:
***When X is Observed***
∑(𝑋 − 𝑋̅)2
𝑉 (𝑋) = 𝐸 [(𝑋 − 𝑋̅)2 ] =
𝑁

𝑉 (𝑋) = 𝐸 [(𝑋 − 𝑋̅ )(𝑋 − 𝑋̅ )′ ]

̂ is Estimated***
***When 𝜷
′
𝑉(𝛽̂ ) = 𝐸 [(𝛽̂ − 𝛽)(𝛽̂ − 𝛽) ]

We know that
(𝑋 ′ 𝑋)−1 = [(𝑋 ′ 𝑋)−1 ]′

Given that:
𝒀 = 𝑋𝛽 + 𝜀 → (1)
𝛽̂ = (𝑋 ′ 𝑋)−1 𝑋 ′ 𝒀 → (2)
′
Derive: 𝐸 [(𝛽̂ − 𝛽)(𝛽̂ − 𝛽) ]

Introduce (1) in (2)

𝛽̂ = (𝑿′ 𝑿)−𝟏 𝑿′ (𝑋𝛽 + 𝜀 )

𝛽̂ = (𝑋 ′ 𝑋)−1 (𝑋 ′ 𝑋)𝛽 + (𝑋 ′ 𝑋)−1 𝑋 ′ 𝜀

𝛽̂ = 𝛽 + (𝑋 ′ 𝑋)−1 𝑋 ′ 𝜀 → (3)
 (𝛽̂ − 𝛽) = (𝑋 ′ 𝑋)−1 𝑋 ′ 𝜀
′
(𝛽̂ − 𝛽)(𝛽̂ − 𝛽) = (𝑋 ′ 𝑋)−1 𝑋 ′ 𝜀 [(𝑋 ′ 𝑋)−1 𝑋 ′ 𝜀]′

′
(𝛽̂ − 𝛽)(𝛽̂ − 𝛽) = (𝑋 ′ 𝑋)−1 𝑋′ 𝜀𝜀
⏟′ 𝑋 (𝑋 ′ 𝑋)−1
𝑀𝑒𝑎𝑡

𝐸 ̂ − 𝛽)(𝛽̂ − 𝛽)′ ] = 𝐸 [(𝑋 ′ 𝑋)−1 𝑋 ′ 𝜀𝜀

[(𝛽 ⏟′ 𝑋(𝑋 ′ 𝑋)−1 ]
⏟
𝑀𝑒𝑎𝑡
̂)
𝑉(𝛽

In Econometrics, the Expected is assumed to join the estimated part

not the observed part

𝑉(𝛽̂𝑂𝐿𝑆 ) = (𝑋 ′ 𝑋)−1 𝑋 ′ 𝐸(𝜀𝜀 ′ )𝑋(𝑋 ′ 𝑋)−1

All the Estimators have an assumed structure for the Expected meat part.
We will report the assumptions here, but we will free the estimators from
those assumptions in Topic Number 4.

𝜀1 𝜀12 𝜀1 𝜀2 𝜀1 𝜀3
𝐸 (𝜀𝜀 ′ ) = 𝐸 [(𝜀2 ) (𝜀1 𝜀2 𝜀3 )1×3 ] = 𝐸 (𝜀1 𝜀2 𝜀22 𝜀3 𝜀2 )
𝜀3 3×1 𝜀1 𝜀3 𝜀3 𝜀2 𝜀32
The Matrix above is called the Variance Covariance Matrix of Errors.

𝐸(𝜀12 ) 𝐸(𝜀1 𝜀2 ) 𝐸(𝜀1 𝜀3 )

𝐸 (𝜀𝜀 ′ ) = (𝐸(𝜀1 𝜀2 ) 𝐸(𝜀22 ) 𝐸(𝜀3 𝜀2 ))
𝐸(𝜀1 𝜀3 ) 𝐸(𝜀3 𝜀2 ) 𝐸(𝜀32 )
The Diagonal is the Variance: Assume the Variance is the Same: Homo-
skedasticity
The Off Diagonal is the Covariance: Assume the Covariances are Zero:
No Autocorrelation
𝜎2 0 0
2
𝐸 (𝜀𝜀 ′ ) = ( 0 𝜎2 0 ) = 𝜎 2 𝐼 = 𝜎1×1
0 0 𝜎2
No Autocorrelation: For Example, a given stock has random steps,
without any pattern, what happened yesterday is unrelated to what is likely
to happen tomorrow

Homo-skedasticity: Variance is the Same. We assume for instance, no

rich and poor people in the same data. People in the data responds in the
same manner.

Introducing the Assumptions above in the Sandwich matrix will result in

the following:
𝑉(𝛽̂𝑂𝐿𝑆 ) = (𝑋 ′ 𝑋)−1 𝑋 ′ 𝜎 2 𝑋(𝑋 ′ 𝑋)−1
Any Scalar can be taken at the beginning of the equation

𝑉(𝛽̂𝑂𝐿𝑆 ) = 𝜎 2 (𝑋 ′ 𝑋)−1 (𝑋 ′ 𝑋)(𝑋 ′ 𝑋)−1

̂ 𝑶𝑳𝑺 ) = 𝝈𝟐 (𝑿′ 𝑿)−𝟏

𝑽(𝜷
2
Econometric Estimators used to assume that 𝐸 (𝜀𝜀 ′ ) = 𝜎1×1 .
We free the Econometric Models from those assumption in chapter 4.
In Applied Exercises:

𝟐 ′)
𝜺′ 𝜺
𝝈 = 𝑬(𝜺𝜺 =
𝒏−𝒌−𝟏
𝑛 = 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑂𝑏𝑠𝑒𝑟𝑣𝑎𝑟𝑡𝑖𝑜𝑛𝑠
𝑘 = 𝑁𝑢𝑚𝑏𝑒𝑟 𝑜𝑓 𝑋𝑠
1 = 𝑓𝑜𝑟 𝑡ℎ𝑒 𝑐𝑜𝑛𝑠𝑡𝑎𝑛𝑡
From the Reference Example, 𝑬𝒔𝒕𝒊𝒎𝒂𝒕𝒆:
(1)𝜀
(2) 𝜀 ′ 𝜀
(3)𝜎 2
(4) 𝑉(𝛽̂ )
(1) The Vector of Errors
𝜀 = 𝑌 − 𝑌̂
0 1 1 0 1.6
2 1 0 0.9 2 0.9
𝜀 = 𝑌 − 𝑋𝛽̂ = ( ) − ( ) ( ) =( )−( )
0 1 −1 0.7 2×1 0 0.2
3 1 2 4×2 3 2.3

−1.6
1.1
𝜀=( )
−0.2
0.7

∑𝜀 = 0
 (2) Sum Squared of Errors
−1.6
1.1
𝜀 ′ 𝜀 = (−1.6 1.1 −0.2 0.7)1×4 ( ) = 4.3
−0.2
0.7 4×1
(3) Variance of Errors (𝜎 2 )

2
𝜀′𝜀 4.3
𝜎 = = = 2.15
𝑛−𝑘−1 4−1−1
(4) Variance of 𝛽̂ (under 𝐸 (𝜀𝜀 ′ ) = 𝜎 2 )
𝑉(𝛽̂ ) = 𝜎 2 (𝑋 ′ 𝑋)−1

1 6 −2 0.645 −0.215
𝑉(𝛽̂ ) = 2.15 × ( ) =( )
20 −2 4 2×2 −0.215 0.43
Variance Covariance Matrix of Parameters

𝑉(𝛽̂1 ) = 0.43 → 𝜎𝛽̂ = √0.43 = 0.6557

𝑉 (𝛼̂ ) = 0.645 → 𝜎𝛼̂ = √0.645 = 0.803

 𝛼̂ = 0.9 → 𝜎𝛼̂ = 0.803
𝛽̂ = 0.7 → 𝑉(𝛽̂ ) = 0.43 → 𝜎𝛽̂ = 0.6557
𝜀 ′ 𝜀 = 4.3
𝜎 2 = 2.15
𝜀′ = (−1.6 1.1 −0.2 0.7)

• Is 𝛽̂ significant or not?
When the Variance is low, 𝛽̂ is more likely to be significant.
However, we need a cutoff point to take this decision.

𝛽̂ − 𝛽
𝛽̂ → 𝑡 =
𝜎𝛽̂

𝐻𝑜 : 𝛽 = 0 → (𝑁𝑜𝑡 𝑆𝑖𝑔)

𝐻1 : 𝛽 ≠ 0 → (𝑆𝑖𝑔)

0.7 − 0
𝑡= = 1.07
0.6557
Compare this value with t table 𝑡𝑛−𝑘−1 → 𝑡2 = 4.3

Given that 𝑡 < 𝑡𝑡𝑎𝑏 : 𝐷𝑜 𝑁𝑜𝑡 𝑅𝑒𝑗𝑒𝑐𝑡 𝐻𝑜

We conclude that 𝛽̂ is not significant!
Important Assumption in OLS as per Econ-3081
𝐶𝑜𝑣 (𝑋, 𝜀 ) = 𝑋 ′ 𝜀 = 0

𝐶𝑜𝑛𝑠 = 𝛼 + 𝛽 𝐼𝑛𝑐𝑜𝑚𝑒 + 𝜀
Everything held constant, when Income increases by 1 USD,
Consumption increase by 0.01 USD.

If X is Correlated with the error term:

(1) X is no longer independent in the model (it is becoming
dependent).
(2) Your interpretation is incorrect (Nothing is held
constant).
(3) Given that 𝜀 = 𝑌 − 𝑌̂, therefore X is affected by Y
(𝑖𝑛𝑣𝑒𝑟𝑠𝑒 𝑐𝑎𝑢𝑠𝑎𝑙𝑖𝑡𝑦).
(4) The 𝛽 you estimated is correlation not regression!

Finding 𝐶𝑜𝑣 (𝑋, 𝜀 ) = 0 is not enough to proof that we don’t have

reverse causality. In practice, two variables might be dependent
on each other and have zero correlation.

(Market Value) (Returns)

X Z Y=ZX
6.062178 1 6.062178
0.866025 1 0.866025
6.062178 -1 -6.06218
0.866025 -1 -0.86603

Correlation(X,Y) 0
 Application:
𝐶𝑜𝑟𝑟𝑢𝑝𝑡𝑖𝑜𝑛 𝐹𝑖𝑟𝑚 𝑆𝑎𝑙𝑒𝑠
(0 − 100)
1 1
0 0
2 4
0 2

(1) Based on OLS Model, Estimate the effect of Corruption on

Firm Sales.
(2) Critically discuss whether this Effect Significant?
(3) Under which assumptions you result are valid?

Recap:
• OLS 𝑀𝑖𝑛 ∑𝜀 2
• For any estimator we can derive and estimate 𝛽̂ , but significance
should be tested and therefore the formula of 𝑉(𝛽̂ ) will be
needed!
• For any Estimator, 𝑉(𝛽̂ ) has two assumptions: No
Autocorrelation, Homoskedasticity.
• OLS Model assumes that 𝐶𝑜𝑣 (𝑋, 𝜀 ) = 0 as we aim to estimate
regression not correlation.
• 𝑪𝒐𝒗(𝑿, 𝜺) might exists in higher order (non-linear) and is less
likely to be captured by simple computations!