Time Series Analysis Lecture 15

Langtian Ma, Yufei Cai, Zhuorang Liu

April 28, 2023

Before beginning the note for lecture 15, we recall some important definitions
in the last lecture:

Definition 1 (Pure seasonal ARMA model) For P, Q ≥ 0, s > 0 which

are all integers, we say {Xt } an ARMA(P, Q)s process if

Φ(B s )Xt = Θ(B s )Wt

PP PQ
where Wt ∼ W N (0, σ 2 ), Φ(B s ) = 1− j=1 Φj B js and Θ(B s ) = 1+ j=1 Θj B js .

Definition 2 (Multiplicative seasonal ARIMA model) For P, Q, p, q ≥ 0,

s > 0 which are all integers, we say {Xt } is an ARMA(p, q) × (P, Q)s process if

Φ(B s )ϕ(B)Xt = Θ(B s )θ(B)Wt

PP PQ
where Wt ∼ W N (0, σ 2 ), Φ(B s ) = 1− j=1 Φj B js and Θ(B s ) = 1+ j=1 Θj B js .
Additionally, for D or d > 0 which are integers, {Xt } is an ARIMA(p, d, q)×
(P, D, Q)s process if

Φ(B s )ϕ(B)∇D d s
s ∇ Xt = Θ(B )θ(B)Wt ,

where ∇ = 1 − B and ∇s = 1 − B s .

Definition 3 (Error of the best prediction) The error of the best predic-
tion X̂t (l) is defined as et (l) ≜ Xt+l − X̂t (l).

1 Forecast Seasonal Models

The first part of lecture 15 gives several examples about forecasting with differ-
ent seasonal models.

Example 1 (ARIMA(0, 1, 1) × (1, 0, 1)12 )

Xt − Xt−1 = Φ(Xt−12 − Xt−13 ) + Wt − θWt−1 − ΘWt−12 + θΘWt−13

where Wt ∼ W N (0, σ 2 ).

1
 Rewrite the model equation to make it a special form of ARMA(13, 13), and
then we get

Xt = Xt−1 + ΦXt−12 − ΦXt−13 + Wt − θWt−1 − ΘWt−12 + θΘWt−13 .

Then the best predictions are:

X̂t (1) =E(Xt+1 |Ft )

=E(Xt + ΦXt−11 − ΦXt−12 + Wt+1 − θWt − ΘWt−11 + θΘWt−12 |Ft )
=Xt + ΦXt−11 − ΦXt−12 − θWt − ΘWt−11 + θΘWt−12
X̂t (2) =E(Xt+2 |Ft )
=E(Xt+1 + ΦXt−10 − ΦXt−11 + Wt+2 − θWt+1 − ΘWt−10
+ θΘWt−11 |Ft )
=X̂t (1) + ΦXt−10 − ΦXt−11 − ΘWt−10 + θΘWt−11
..
.
X̂t (l) =E(Xt+l |Ft )
=E(Xt+l−1 + ΦXt+l−12 − ΦXt+l−13 + Wt+l − θWt+l−1 − ΘWt+l−12
+ θΘWt+l−13 |Ft )
=X̂t (l − 1) + ΦX̂t (l − 12) − ΦX̂t (l − 13) if l > 13

Example 2 (AR(1)12 (⊆AR(12))) Xt = ΦXt−12 +Wt where Wt ∼ W N (0, σ 2 ).

The best prediction is

X̂t (l) =E(Xt+l |Ft )

=E(Xt+l−12 + Wt+l |Ft )
(
ΦXt+l−12 , 1 ≤ l ≤ 12,
=
ΦX̂t (l − 12), l > 12.

=⇒ X̂t (l) = Φk+1 Xt+r−11 where l = 12k + r + 1, 0 ≤ r < 12 which can be

proved by trivial induction.
The variance of the error of the best prediction is given by

1 − Φ2k+2 2
V ar(et (l)) = σ .
1 − Φ2
Proof First we calculate the auto-covariance function:

γ(0) = V ar(ΦXt−12 + Wt ) = Φ2 γ(0) + σ 2

γ(12k) = Cov(Φk Xt , Xt ) = Φk γ(0)

2
 Then the variance of the error follows:
V ar(et (l)) =V ar(Xt+l ) + V ar(X̂t (l)) − 2Cov(Xt+l , Φk+1 Xt+r−11 )
=γ(0) + Φ2k+2 γ(0) − 2Φk+1 γ(l − r + 11)
=(1 + Φ2k+2 )γ(0) − 2Φk+1 γ(12(k + 1))
=(1 + Φ2k+2 − 2Φ2k+2 )γ(0)
1 − Φ2k+2 2
= σ
1 − Φ2
2

Example 3 (MA(1)12 (⊆MA(12))) Xt = Wt +ΘWt−12 where Wt ∼ W N (0, σ 2 ).

X̂t (l) =E(Xt+l |Ft )

=E(Wt+l + ΘWt+l−12 |Ft )
(
ΘWt+l−12 , 1 ≤ l ≤ 12,
=
0, l > 12.
The error of the best prediction is given by
(
σ2 , 1 ≤ l ≤ 12,
V ar(et (l)) = 2 2
(1 + Θ )σ , l > 12.
Proof
et (l) =Xt+l − X̂t (l)
=Wt+l + ΘWt+l−12 − X̂t (l)
(
Wt+l , 1 ≥ l ≤ 12,
=
Wt+l + ΘWt+l−12 , l > 12.
(
σ2 , 1 ≤ l ≤ 12,
V ar(et (l)) =
(1 + Θ2 )σ 2 , l > 12.
2

Exercise 1 Consider a time series Xt = 21 Xt−4 + Wt − θ1 Wt−1 − θ2 Wt−2 where

Wt ∼ W N (0, σ 2 ).
1. Find the first four ψ-weights (causal representation) for this model.
2. Suppose θ1 = 21 , θ2 = − 14 , σ 2 = 1. Find the forecasts for the next four
quarters if the last four quarters are in Table 1.
3. Give the 95% prediction intervals for part 2.

Solution 1 See appendix for the solution.

3
 Q I (t − 3) II (t − 2) III (t − 1) IV (t)
Series (Xt ) 25 20 25 40
Residual (Wt ) 2 1 2 3

Table 1: Last Four Quarters.

2 Models for Seasonality

In some time series, there are obvious periodic changes. This cycle is due to
seasonal variations or some other inherent factor. Such sequences are called
seasonal sequences. One of the models describing such series is the seasonal
time series model.
Here we discuss some models for strong seasonality and models for both
trend and seasonal patterns.

2.1 Models for Strong Seasonality

The basic form of the seasonality only model is

Xt = St + Zt ,

where St is strong seasonality (St+d = St , where d is the period of the seasonality

model) and Zt is stationary.
The following are some examples of this kind of models:

Example 4 (Fourier series) The Fourier series model for the seasonal time
series is
k
X f f
St = a0 + [(af sin(2π t) + bf cos(2π t)],
d d
f =1

where af and bf are the amplitude of St , f /d and d/f are the frequency and
period of the sin and cos part, respectively.

Remark This function is actually the first k terms of a Fourier Series, see
section 3 for more details.

Example 5 (Non-parametric seasonality model) For Si , Let

1
Sbi = (Si + Si+d + · · · + Si+kd ),
k+1
where d is the period of St .

Remark In this model, small sample size can easily lead to over-fitting.

4
 Example 6 (Differencing) In some applications, seasonality is secondary and
we need to eliminate it from the data. This process is called seasonal adjustment,
and seasonal differences are a common method. If a sequence is periodic and
the period is d, then the seasonal differential is

Xt − Xt−d = St − St−d + Zt − Zt−d = Zt − Zt−d .

By seasonal differencing, we can get a stationary sequence Zt − Zt−d .

2.2 Models for Trend and Seasonality

Next, we discuss some models which include both trend and seasonality patterns.
If a model have both trend and seasonality, the basic form of it is

Xt = Mt + St + Zt .

Where Mt is the trend, St is the seasonal pattern and Zt is the stationary

pattern. The following are some alternatives to model this kind of time series.

Example 7 (Parametric model) First we fit a linear regression model to the

data to capture its trend pattern, write:

Mt = β0 + β1 t.

For the remaining part, we can use Fourier series model in Example 4 to fit it.

Example 8 (Smoothing) Alternatively, we can use smoothing to fit the trend

part of the data. Note that we need to average all observations in a whole period.
Then the trend estimation at time t is:
1
m̂t = (Xt−d/2 + Xt−d/2+1 + ... + Xt+d/2 )
d
Remark To illustrate why we need to average over all observations in a whole
period, we use AirPassengers data set in R as an example.
1 library ( zoo )
2 library ( forecast )
3 data ( " AirPassengers " )
4 passengers <- ts ( AirPassengers , start = c (1949 , 1) , frequency = 12)
5
6 # Average over the whole period .
7 Homo _ fit <- rollmean ( passengers , k = 12)
8 # Avergae over part of a period .
9 Hetero _ fit <- rollmean ( passengers , k = 7)
10
11 autoplot ( passengers ) +
12 autolayer ( Homo _ fit , color = " blue " ) +
13 autolayer ( Hetero _ fit , color = " red " )

5
 Figure 1: Fitting with smoothing

In figure 1, the black line indicates the real data. The blue line is the
smoothing result with k = d = 12 (the period of the data observed by the plot).
The red line is the smoothing result with k = 7. We can observe that the red
line oscillates more frequently than the blue line, which means that the model
with k = 7 does not capture the trend pattern well.

Example 9 (Differencing) We can also remove the seasonal term by the sea-
sonal difference:

Xt − Xt−d =Mt − Mt−d + St − St−d + Zt − Zt−d

=Mt − Mt−d + Zt − Zt−d .

Then the differenced time series is a non-seasonal time series with trend pattern.
We can use ARIMA model to fit it.

Remark Note that to some extend, this is equivalent to modeling with multi-
plicative seasonal ARIMA model. But In this manner we may directly obtain
d by observation or prior knowledge, which can lead to a simpler model.

3 More on Fourier Series

The model in example 4 is actually first k terms of a Fourier Series. Here we
provide some further discussion.
A Fourier series is an expansion of periodic function into a sum of trigono-
metric functions. It can be represented in different forms, here we introduce the
sine-cosine form of Fourier series:

6
Definition 4 For a periodic function s(x), its Fourier series can be written as:
∞    
X 2πnx 2πnx
s(x) ∼ A0 + An cos + Bn sin ,
n=1
P P

where the Fourier series coefficients are defined by the integrals:

Z P/2
1
A0 = s(x)dx
P −P/2
Z P/2  
2 2πnx
An = s(x) cos dx for n ≥ 1
P −P/2 P
Z P/2  
2 2πnx
Bn = s(x) sin dx for n ≥ 1
P −P/2 P

It can be shown that a Fourier series is a valid representation of any periodic

function. For a pure seasonal time series {Yt }nt=1 , we only take advantage of
the first k terms to depict the seasonality, as shown in example 4, we have the
model:
k
X f f
Yt = a0 + [(af sin(2π t) + bf cos(2π t)].
d d
f =1

Ordinary least squares regression can be used to fit the a’s and b’s. For odd
n, write n = 2k + 1, we have the estimations:

â0 = Ȳ
n n
2X 2X
âj = Yt cos(2πtj/n) and b̂j = Yt sin(2πtj/n).
n t=1 n t=1

For even n, say n = 2k, the equations above still apply for j = 1, 2, ..., k − 1,
but
n
1X
âk = (−1)t Yt and b̂k = 0.
n t=1

7
Appendices
Solution to Exercise 1
1.
1
Xt = Xt−4 + Wt − θ1 Wt−1 − θ2 Wt−2
2
1 1
= ( Xt−8 + Wt−4 − θ1 Wt−5 − θ2 Wt−6 ) + Wt − θ1 Wt−1 − θ2 Wt−2
2 2
1 1 1 1
=Wt − θ1 Wt−1 − θ2 Wt−2 + Wt−4 − θ1 Wt−5 − θ2 Wt−6 + Xt−8
2 2 2 4

So ψ0 = 1, ψ1 = −θ1 , ψ2 = −θ2 , ψ3 = 0, ψ4 = 12 .
2.

X̂t (1) =E(Xt+1 |Ft )

1
= Xt−3 − θ1 Wt − θ2 Wt−1
2
1 1 1
= × 25 − × 3 + × 2
2 2 4
23
=
2
X̂t (2) =E(Xt+2 |Ft )
1
= Xt−2 − θ2 Wt
2
1 1
= × 20 + × 3
2 4
43
=
4
X̂t (3) =E(Xt+3 |Ft )
1
= Xt−1
2
1
= × 25
2
25
=
2
X̂t (4) =E(Xt+4 |Ft )
1
= Xt
2
1
= × 40
2
=20

8
3.

et (1) =Xt+1 − X̂t (1)

1 1
= Xt−3 + Wt+1 − θ1 Wt − θ2 Wt−1 − ( Xt−3 − θ1 Wt − θ2 Wt−1 )
2 2
=Wt+1
=⇒ V ar(et (1)) = σ 2 = 1
et (2) =Xt+2 − X̂t (2)
1 1
= Xt−2 + Wt+2 − θ1 Wt+1 − θ2 Wt − ( Xt−2 − θ2 Wt )
2 2
=Wt+2 − θ1 Wt+1
5
=⇒ V ar(et (2)) = (1 + θ12 )σ 2 =
4
et (3) =Xt+3 − X̂t (3)
1 1
= Xt−1 + Wt+3 − θ1 Wt+2 − θ2 Wt+1 − Xt−1
2 2
=Wt+3 − θ1 Wt+2 − θ2 Wt+1
21
=⇒ V ar(et (3)) = (1 + θ12 + θ22 )σ 2 =
16
et (4) =Xt+4 − X̂t (4)
1 1
= Xt + Wt+4 − θ1 Wt+3 − θ2 Wt+2 − Xt
2 2
=Wt+4 − θ1 Wt+3 − θ2 Wt+2
21
=⇒ V ar(et (4)) = (1 + θ12 + θ22 )σ 2 =
16
p
So the prediction interval for X̂t (i) is X̂t (i) ± zα/2 V ar(et (i)) for any
i ∈ {1, 2, 3, 4} where zα/2 is the upper α quantile of the standard normal
distribution.