Chapter 6

Univariate time series modelling and forecasting

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 1

 Univariate Time Series Models

Where we attempt to predict returns using only information contained

in their past values.

Some Notation and Concepts

A Strictly Stationary Process

A strictly stationary process is one where

P{yt1 ≤ b1 , . . . , ytn ≤ bn } = P{yt1 +m ≤ b1 , . . . , ytn +m ≤ bn }

A Weakly Stationary Process

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 2

 Univariate Time Series Models (Cont’d)
If a series satisfies the next three equations, it is said to be weakly or
covariance stationary

(1) E (yt ) = µ t = 1, 2, . . . , ∞
(2) E (yt − µ)(yt − µ) = σ 2 < ∞
(3) E (yt1 − µ)(yt2 − µ) = γt2 −t1 ∀ t1 , t2

So if the process is covariance stationary, all the variances are the

same and all the covariances depend on the difference between t1 and
t2 . The moments

E (yt − E (yt ))(yt−s − E (yt−s )) = γs , s = 0, 1, 2, . . .

are known as the covariance function.

The covariances, γs , are known as autocovariances.

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 3

 Univariate Time Series Models (Cont’d)

However, the value of the autocovariances depend on the units of

measurement of yt .
It is thus more convenient to use the autocorrelations which are the
autocovariances normalised by dividing by the variance:
γs
τs = , s = 0, 1, 2, . . .
γ0

If we plot τs against s=0,1,2,... then we obtain the autocorrelation

function or correlogram.

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 4

 A White Noise Process

A white noise process is one with (virtually) no discernible structure.

A definition of a white noise process is

E (yt ) = µ
var(yt ) = σ 2
 2
σ if t = r
γt−r =
0 otherwise

Thus the autocorrelation function will be zero apart from a single

peak of 1 at s=0. τ̂s ∼ approx. N(0, 1/T ) where T = sample size

We can use this to do significance tests for the autocorrelation

coefficients by constructing a confidence interval.

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 5

 A White Noise Process (Cont’d)

For example, a 95 % confidence interval would be given by

1
±1.96 × √
T

If the sample autocorrelation coefficient, τ̂s , falls outside this region

for any value of s, then we reject the null hypothesis that the true
value of the coefficient at lag s is zero.

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 6

 Joint Hypothesis Tests
We can also test the joint hypothesis that all m of the τk correlation
coefficients are simultaneously equal to zero using the Q-statistic
developed by Box and Pierce:
m
X
Q=T τ̂k2
k=1

where T=sample size, m=maximum lag length

The Q-statistic is asymptotically distributed as a χ2m .
However, the Box Pierce test has poor small sample properties, so a
variant has been developed, called the Ljung-Box statistic:
m
∗
X τ̂k2
Q = T (T + 2) ∼ χ2m
T −k
k=1

This statistic is very useful as a portmanteau (general) test of linear

dependence in time series.
‘Introductory Econometrics for Finance’ c Chris Brooks 2013 7
 An ACF Example

Question:
Suppose that a researcher had estimated the first 5 autocorrelation
coefficients using a series of length 100 observations, and found them
to be (from 1 to 5): 0.207, -0.013, 0.086, 0.005, -0.022.
Test each of the individual coefficient for significance, and use both
the Box-Pierce and Ljung-Box tests to establish whether they are
jointly significant.
Solution:
A coefficient would be significant if it lies outside (-0.196,+0.196) at
the 5% level, so only the first autocorrelation coefficient is significant.
Q=5.09 and Q*=5.26
Compared with a tabulated χ2 (5)=11.1 at the 5% level, so the 5
coefficients are jointly insignificant.

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 8

 Moving Average Processes
Let ut (t = 1, 2, 3, . . . ) be a sequence of independently and
identically distributed (iid) random variables with E(ut ) = 0 and
var(ut ) = σ 2 , then

yt = µ + ut + θ1 ut−1 + θ2 ut−2 + · · · + θq ut−q

is a qth order moving average model MA(q).

Its properties are

E(yt ) = µ
var(yt ) = γ0 = 1 + θ12 + θ22 + · · · + θq2 σ 2

Covariances
(
(θs + θs+1 θ1 + θs+2 θ2 + · · · + θq θq−s ) σ 2 for s = 1, . . . , q
γs =
0 for s > q

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 9

 Example of an MA Problem

1 Consider the following MA(2) process:

yt = ut + θ1 ut−1 + θ2 ut−2

where ut is a zero mean white noise process with variance σ 2 .

i. Calculate the mean and variance of Xt
ii. Derive the autocorrelation function for this process (i.e. express the
autocorrelations, τ1 , τ2 , ...as functions of the parameters θ1 and θ2 ).
iii. If θ1 = −0.5 and θ2 = 0.25, sketch the acf of Xt .

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 10

 Solution

i. If E (ut ) = 0, then E(ut−i ) = 0 ∀ i So

E(yt ) = E(ut + θ1 ut−1 + θ2 ut−2 )

= E(ut ) + θ1 E(ut−1 ) + θ2 E(ut−2 ) = 0
var(yt ) = E[yt − E(yt )][yt − E(yt )]

But E(yt ) = 0, so

var(yt ) = E[(yt )(yt )]

var(yt ) = E[(ut + θ1 ut−1 + θ2 ut−2 )(ut + θ1 ut−1 + θ2 ut−2 )]

2
var(yt ) = E ut2 + θ12 ut−1 + θ22 ut−2
2
 
+ cross-products

But E[cross-products] = 0 since cov(ut , ut−s ) = 0 for s 6= 0.

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 11

 Solution (Cont’d)

var(yt ) = γ0 = E ut2 + θ12 ut−1

2
+ θ22 ut−2
2
 
So

= σ 2 + θ12 σ 2 + θ22 σ 2

= 1 + θ12 + θ22 σ 2

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 12

 Solution (Cont’d)

ii. The acf of yt

γ1 = E[yt − E(yt )][yt−1 − E(yt−1 )]

γ1 = E[yt ][yt−1 ]

γ1 = E[(ut + θ1 ut−1 + θ2 ut−2 )(ut−1 + θ1 ut−2 + θ2 ut−3 )]

2 2


γ1 = E θ1 ut−1 + θ1 θ2 ut−2

γ 1 = θ 1 σ 2 + θ1 θ2 σ 2

γ1 = (θ1 + θ1 θ2 )σ 2

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 13

 Solution (Cont’d)

γ2 = E[yt − E(yt )][yt−2 − E(yt−2 )]

γ2 = E[yt ][yt−2 ]
γ2 = E[(ut + θ1 ut−1 + θ2 ut−2 )(ut−2 + θ1 ut−3 + θ2 ut−4 )]
2


γ2 = E θ2 ut−2
γ2 = θ 2 σ 2

γ3 = E[yt − E(yt )][yt−3 − E(yt−3 )]

γ3 = E[yt ][yt−3 ]

γ3 = E[(ut + θ1 ut−1 + θ2 ut−2 )(ut−3 + θ1 ut−4 + θ2 ut−5 )]

γ3 = 0
‘Introductory Econometrics for Finance’ c Chris Brooks 2013 14
 Solution (Cont’d)
So γs = 0 for s > 2.
We have the autocovariances, now calculate the autocorrelations:

γ0
τ0 = =1
γ0
γ1 (θ1 + θ1 θ2 )σ 2 (θ1 + θ1 θ2 )
τ1 = = 2 2

=
1 + θ12 + θ22


γ0 1 + θ1 + θ2 σ 2

γ2 (θ2 )σ 2 θ2
τ2 = = 2 2

=
1 + θ12 + θ22


γ0 1 + θ1 + θ2 σ 2

γ3
τ3 = =0
γ0
γs
τs = =0 ∀ s>2
γ0

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 15

 Solution (Cont’d)

iii. For θ1 = −0.5 and θ2 = 0.25, substituting these into the formulae
above gives τ1 = −0.476, τ2 = 0.190.

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 16

 ACF Plot
Thus the acf plot will appear as follows:
1.2

0.8

0.6

0.4
acf

0.2

0
0 1 2 3 4 5
–0.2

–0.4

–0.6

lag, s

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 17

 Autoregressive Processes
An autoregressive model of order p, an AR(p) can be expressed as

yt = µ + φ1 yt−1 + φ2 yt−2 + · · · + φp yt−p + ut

Or using the lag operator notation:

Lyt = yt−1 Li yt = yt−i

p
X
yt = µ + φi yt−i + ut
i=1

or
p
X
yt = µ + φi Li yt + ut
i=1

or φ(L)yt = µ + ut where φ(L) = (1 − φ1 L − φ2 L2 − · · · − φp Lp ).

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 18
 The Stationarity Condition for an AR Model

The condition for stationarity of a general AR( p) model is that the

roots of 1 − φ1 z − φ2 z 2 − · · · − φp z p = 0 all lie outside the unit
circle.

A stationary AR(p) model is required for it to have an MA(∞)

representation.

Example 1: Is yt = yt−1 + ut stationary?

The characteristic root is 1, so it is a unit root process (so
non-stationary)

Example 2: Is yt = 3yt−1 − 2.75yt−2 + 0.75yt−3 + ut stationary?

The characteristic roots are 1, 2/3, and 2. Since only one of these lies
outside the unit circle, the process is non-stationary.

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 19

 The Moments of an Autoregressive Process
The moments of an autoregressive process are as follows. The mean
is given by
φ0
E (yt ) =
1 − φ1 − φ2 − · · · − φp

The autocovariances and autocorrelation functions can be obtained

by solving what are known as the Yule-Walker equations:

τ1 = φ1 + τ1 φ2 + · · · + τp−1 φp
τ2 = τ1 φ1 + φ2 + · · · + τp−2 φp
.. .. ..
. . .
τp = τp−1 φ1 + τp−2 φ2 + · · · + φp

If the AR model is stationary, the autocorrelation function will decay

exponentially to zero.
‘Introductory Econometrics for Finance’ c Chris Brooks 2013 20
 Sample AR Problem

Consider the following simple AR(1) model

yt = µ + φ1 yt−1 + ut

i. Calculate the (unconditional) mean of yt .

For the remainder of the question, set µ = 0 for simplicity.
ii. Calculate the (unconditional) variance of yt .
iii. Derive the autocorrelation function for yt .

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 21

 Solution

i. Unconditional mean:

E(yt ) = E(µ + φ1 yt−1 )

E(yt ) = µ + φ1 E(yt−1 )

But also

E(yt ) = µ + φ1 (µ + φ1 E(yt−2 ))
= µ + φ1 µ + φ21 E(yt−2 )
= µ + φ1 µ + φ21 (µ + φ1 E(yt−3 ))

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 22

 Solution (Cont’d)
So

E(yt ) = µ + φ1 (µ + φ1 E(yt−2 ))
= µ + φ1 µ + φ21 E(yt−2 )
= µ + φ1 µ + φ21 (µ + φ1 E(yt−3 ))
E(yt ) = µ + φ1 µ + φ21 µ + φ31 E(yt−3 )

An infinite number of such substitutions would give

E(yt ) = µ 1 + φ1 + φ21 + · · · ) + φ∞
1 y0

So long as the model is stationary, i.e. |φ1 | < 1, then φ∞

1 = 0.
So
µ
E(yt ) = µ 1 + φ1 + φ21 + · · · =


1 − φ1
‘Introductory Econometrics for Finance’ c Chris Brooks 2013 23
 Solution (Cont’d)

ii. Calculating the variance of yt : yt = φ1 yt−1 + ut

From Wold’s decomposition theorem:

yt (1 − φ1 L) = ut
yt = (1 − φ1 L)−1 ut
1 + φ1 L + φ21 L2 + · · · ut


yt =

So long as, |φ1 | < 1, this will converge.

var(yt ) = E[yt − E(yt )][yt − E(yt )]

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 24

 Solution (Cont’d)
but E(yt ) = 0, since µ is set to zero.

var(yt ) = E[(yt )(yt )]

= E ut + φ1 ut−1 + φ21 ut−2 + · · · ut + φ1 ut−1


+φ21 ut−2 + · · ·



= E ut2 + φ21 ut−1

2
+ φ41 ut−2
2
 
+ · · · + cross-products
= E ut2 + φ21 ut−1
2
+ φ41 ut−2
2
 
+ ···
= σu2 + φ21 σu2 + φ41 σu2 + · · ·
= σu2 1 + φ21 + φ41 + · · ·

σu2
=
(1 − σu2 )

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 25

 Solution (Cont’d)
iii. Turning now to calculating the acf, first calculate the autocovariances:
γ1 = cov (yt , yt−1 ) = E[yt − E (yt )][yt−1 − E (yt−1 )]
Since a0 has been set to zero, E(yt ) = 0 and E(yt−1 ) = 0, so

γ1 = E[yt yt−1 ]
under the result above that E(yt ) = E(yt−1 ) = 0. Thus
γ1 = E ut + φ1 ut−1 + φ21 ut−2 + · · · ut−1 + φ1 ut−2


+ φ21 ut−3 + · · ·



2
+ φ31 ut−2
2
 
γ1 = E φ1 ut−1 + · · · + cross − products
γ1 = φ1 σ 2 + φ31 σ 2 + φ51 σ 2 + · · ·
φ1 σ 2
γ1 =
1 − φ21

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 26

 Solution (Cont’d)
For the second autocorrelation coefficient,

γ2 = cov(yt , yt−2 ) = E[yt − E(yt )][yt−2 − E(yt−2 )]

Using the same rules as applied above for the lag 1 covariance

γ2 = E[yt yt−2 ]
γ2 = E ut + φ1 ut−1 + φ21 ut−2 + · · · ut−2 + φ1 ut−3


+ φ21 ut−4 + · · ·



γ2 = E φ21 ut−2
2
+ φ41 ut−3
2
 
+ · · · +cross-products
γ2 = φ21 σ 2 + φ41 σ 2 + · · ·
γ2 = φ21 σ 2 1 + φ21 + φ41 + · · ·

φ21 σ 2
γ2 =
1 − φ21

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 27

 Solution (Cont’d)

If these steps were repeated for γ3 , the following expression would be

obtained

φ31 σ 2
γ3 =
1 − φ21

and for any lag s, the autocovariance would be given by

φs1 σ 2
γs =
1 − φ21

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 28

 Solution (Cont’d)
The acf can now be obtained by dividing the covariances by the
variance:
γ0
τ0 = =1
γ0
!
φ1 σ 2
1 − φ21


γ1
τ1 = = ! = φ1
γ0 σ2
1 − φ21


!
φ21 σ 2
1 − φ21


γ2
τ2 = = ! = φ21
γ0 σ 2

1 − φ21

τ3 = φ31
τs = φs1
‘Introductory Econometrics for Finance’ c Chris Brooks 2013 29
 The Partial Autocorrelation Function (denoted τkk )
Measures the correlation between an observation k periods ago and
the current observation, after controlling for observations at
intermediate lags (i.e. all lags <k).

So τkk measures the correlation between yt and yt−k after removing

the effects of yt−k+1 , yt−k+2 , . . . , yt−1

At lag 1, the acf = pacf always

At lag 2,

τ22 = τ2 − τ12 1 − τ12



For lags 3+, the formulae are more complex.

The pacf is useful for telling the difference between an AR process

and an ARMA process.

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 30

 The Partial Autocorrelation Function (denoted τkk )
(Cont’d)

In the case of an AR(p), there are direct connections between yt and

yt−s only for s ≤ p.

So for an AR(p), the theoretical pacf will be zero after lag p.

In the case of an MA(q), this can be written as an AR(∞), so there

are direct connections between yt and all its previous values.

For an MA(q), the theoretical pacf will be geometrically declining.

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 31

 ARMA Processes
By combining the AR(p) and MA(q) models, we can obtain an
ARMA(p,q) model:

φ(L)yt = µ + θ(L)ut

where

φ(L) = 1 − φ1 L − φ2 L2 − · · · − φp Lp and

θ(L) = 1 + θ1 L + θ2 L2 + · · · + θq Lq

or

yt = µ + φ1 yt−1 + φ2 yt−2 + · · · + φp yt−p + θ1 ut−1

+ θ2 ut−2 + · · · + θq ut−q + ut

with
E(ut ) = 0; E ut2 = σ 2 ; E (ut us ) = 0, t 6= s

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 32

 Summary of the Behaviour of the acf for AR and
MA Processes

An autoregressive process has

a geometrically decaying acf

number of spikes of pacf = AR order

A moving average process has

Number of spikes of acf = MA order

a geometrically decaying pacf

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 33

 Some sample acf and pacf plots for standard
processes
The acf and pacf are not produced analytically from the relevant
formulae for a model of that type, but rather are estimated using
100,000 simulated observations with disturbances drawn from a
normal distribution.

Figure: Sample autocorrelation and partial autocorrelation functions for an

MA(1) model: yt = −0.5ut−1 + ut
0.05

0
1 2 3 4 5 6 7 8 9 10
–0.05

–0.1

–0.15
acf and pacf

–0.2

–0.25

–0.3
acf
–0.35 pacf

–0.4

–0.45
lag, s
‘Introductory Econometrics for Finance’ c Chris Brooks 2013 34
 ACF and PACF for an MA(2) Model:
yt = 0.5ut−1 − 0.25ut−2 + ut
0.4

0.3 acf
pacf
0.2

0.1
acf and pacf

0
1 2 3 4 5 6 7 8 9 10

–0.1

–0.2

–0.3

–0.4
lag, s

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 35

 ACF and PACF for a slowly decaying AR(1) Model:
yt = 0.9yt−1 + ut

0.9
acf
0.8
pacf
0.7

0.6
acf and pacf

0.5

0.4

0.3

0.2

0.1

0
1 2 3 4 5 6 7 8 9 10
–0.1
lag, s

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 36

 ACF and PACF for a more rapidly decaying AR(1)
Model: yt = 0.5yt−1 + ut

0.6

0.5

acf
0.4 pacf
acf and pacf

0.3

0.2

0.1

0
1 2 3 4 5 6 7 8 9 10

–0.1
lag, s

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 37

 ACF and PACF for a more rapidly decaying AR(1)
Model with Negative Coefficient: yt = −0.5yt−1 + ut

0.3

0.2

0.1

0
1 2 3 4 5 6 7 8 9 10
acf and pacf

–0.1

–0.2

–0.3

–0.4 acf
pacf
–0.5

–0.6
lag, s

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 38

 ACF and PACF for a Non-stationary Model (i.e. a
unit coefficient):yt = yt−1 + ut

0.9

0.8

0.7

0.6
acf and pacf

0.5

0.4

0.3
acf
0.2 pacf
0.1

0
1 2 3 4 5 6 7 8 9 10
lag, s

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 39

 ACF and PACF for an ARMA(1,1):
yt = 0.5yt−1 + 0.5ut−1 + ut
0.8

0.6
acf
pacf
0.4
acf and pacf

0.2

0
1 2 3 4 5 6 7 8 9 10

–0.2

–0.4
lag, s

‘Introductory Econometrics for Finance’ c Chris Brooks 2013 40