Chapter 12: Time Series Models of Heteroscedasticity

I Our ARIMA models that we have studied have modeled the

conditional mean of our time series: The mean of Yt given
the previous observations.
I Our ARIMA models have assumed that the conditional
variance is constant and equal to the noise variance, σ 2 .
I For example, our AR(1) model assumes that:
E (Yt |Yt−1 , Yt−2 , . . .) = φYt−1 and
var (Yt |Yt−1 , Yt−2 , . . .) = var (et ) = σ 2
I If our time series exhibits nonconstant variance, sometimes
this can be remedied by transforming the data (for example,
by working with log(Yt )).
I But in other data sets, especially financial time series, the
conditional variance is nonconstant in some irregular, random
pattern that cannot be remedied by a transformation.
I A data set with nonconstant variance is called heteroscedastic.
Hitchcock STAT 520: Forecasting and Time Series
The Return Series {rt }

I In financial data, the return is a kind of growth rate of the

series.
I If Yt is the value or price of an asset at time t, then the
return (also called the relative gain) of the asset at time t is
denoted rt and is defined as
Yt − Yt−1
rt = .
Yt−1
I Note that rt = Yt /Yt−1 − 1, and so 1 + rt = Yt /Yt−1 . Thus
 
Yt
log(1 + rt ) = log
Yt−1
= log(Yt ) − log(Yt−1 ) = ∇ log(Yt )

Hitchcock STAT 520: Forecasting and Time Series

Approximating the Return Series

I And if rt is a very small number (near 0), it turns out that

rt ≈ log(1 + rt ) ≈ log(1) = 0 so we can approximate rt by
∇ log(Yt ).
I This approximation is often done, and sometimes ∇ log(Yt ) is
itself called the return.
I In addition, the return is sometimes multiplied by 100 so that
it can be interpreted as the percent change in price.

Hitchcock STAT 520: Forecasting and Time Series

An Example of a Heteroscedastic Time Series

I See the time series plot of the daily CREF stock values from
August 26, 2004 until August 15, 2006.
I The time series plot shows an increasing trend, and possibly
nonconstant variance.
I A time series plot of the returns 100 × ∇ log(Yt ), t = 1, . . . , n
shows that at certain time periods, the stock price is more
volatile (more variable) than in other time periods, and “quiet
periods” tend to alternate with “volatile periods.”
I This phenomenon is called volatility clustering.
I There is particular volatility near the end of the plot (July
12-August 14, 2006), when there was a war in Lebanon.

Hitchcock STAT 520: Forecasting and Time Series

More Analysis of the CREF Time Series

I An initial investigation of the returns indicates they could be

modeled as white noise.
I Neither the ACF nor PACF indicate significant
autocorrelation, and the mean of the return process is not
significantly different from zero.
I However, the volatility clustering indicates that the variance is
not constant over time, so the returns may not be
independent and identically distributed (iid).
I If the data are iid, then nonlinear transformations of the data
should resemble white noise as well.
I We can take the squared returns or absolute returns, and if
these have significant autocorrelation, this is evidence that the
original data were not iid.

Hitchcock STAT 520: Forecasting and Time Series

Detailed Analysis of the CREF Time Series

I A glance at the ACF and the PACF of the absolute returns of

the CREF series shows there are significant autocorrelations at
several lags (not a white noise-like pattern).
I A look at the ACF and the PACF of the squared returns of
the CREF series tells a similar story.
I The McLeod-Li test is a version of the Ljung-Box test for
autocorrelation based on the squared data.
I The alternative hypothesis is that the data have
autoregressive conditional heteroscedasticity (ARCH).

Hitchcock STAT 520: Forecasting and Time Series

Testing for ARCH in the CREF Time Series

I Under the null hypothesis of no ARCH, the test statistic has a

chi-square distribution with degrees of freedom equal to the
number of autocorrelations used in the test.
I The McLeod.Li.test function plots the p-values of this test
for a variety of different lags.
I For the CREF data, the test is significant when there are more
than 3 lags used, indicating that ARCH does exist for these
data.

Hitchcock STAT 520: Forecasting and Time Series

The ARCH(1) Model

I The simplest model for ARCH data is the ARCH(1) model.

I 2
Let σt|t−1 denote the conditional variance (or conditional
volatility) of rt , given all returns though time t − 1.
I Then the ARCH(1) model for the return process {rt } is:

rt = σt|t−1 t
2 2
σt|t−1 = ω + αrt−1

where α ≥ 0 and ω ≥ 0 are unknown parameters, and the i ’s

are iid random variables with mean zero and variance 1.

Hitchcock STAT 520: Forecasting and Time Series

More on the ARCH(1) Model

I The conditional distribution of rt |rt−1 has mean zero and

2 .
variance ω + αrt−1
I So we see the conditional variance of rt is not constant, and it
depends on the previous return.
I Otherwise, the returns follow a type of white noise process
(that has nonconstant conditional variance).
I A more general ARCH(q) model is possible:

rt = σt|t−1 t
2 2 2
σt|t−1 = ω + α1 rt−1 + · · · + αq rt−q ,

but we will not cover this in detail.

Hitchcock STAT 520: Forecasting and Time Series

Properties of the ARCH(1) Model

I The ARCH(1) model for rt can be written as an AR(1) model

for the squared returns rt2 , where the noise process is
non-normal:

rt2 = ω + αrt−1
2 2
+ σt|t−1 (2t − 1)

I Therefore, we can specify an ARCH(1) model for rt if our

tools suggest an AR(1) model for the squared returns rt2 .
I Also, it can be shown that E (rt ) = 0, and
var (rt ) = σ 2 = ω/(1 − α).
I This implies that 0 ≤ α < 1, if (and only if) the return series
is weakly stationary.

Hitchcock STAT 520: Forecasting and Time Series

Properties of the ARCH(1) Model

I Oddly, the ARCH(1) model is weakly stationary even though

the conditional variance is not constant (but note the
unconditional variance is constant).
I Also, for any h > 0, cov (rt+h , rt ) = 0, so {rt } has zero
autocorrelation at lags greater than 0.
I Finally, the kurtosis of rt is always greater than that of a
normal distribution, so the distribution of rt has “fatter tails”
than a normal.

Hitchcock STAT 520: Forecasting and Time Series

Estimating the ARCH(1) Model

I 2
The conditional variance σt|t−1 is a parameter and is not
2 2
observable, but note that rt is an unbiased estimator of σt|t−1 .
I The parameters ω and α of the ARCH(1) model can be
estimated by conditional ML.
I The garch function in the tseries package can estimate the
ARCH(1) model on real data.
I One issue is that the ARCH likelihood tends to be fairly flat
unless n is large, so it can be difficult for numerical methods
to find the true maximum.

Hitchcock STAT 520: Forecasting and Time Series

Example with a Simulated ARCH(1) Time Series

I See the R example for the plot of the simulated ARCH(1)

series with ω = 0.01, α = 0.9.
I We can clearly see the volatility clustering and nonconstant
variance.
I The garch function estimates ω and α to produce the fitted
2
model for σt|t−1 .
I The diagnostic tests show that the model residuals appear
uncorrelated.
I Note that we may wish to forecast this conditional variance h
time units into the future.
I 2
It can be shown that σt+h|t 2
= ω + ασt+h−1|t , where we let
2 2
σt+h|t = rt+h for h < 0.
I Plugging in the estimates for ω and α, we then have a
recursive formula for the forecasted conditional variance.
Hitchcock STAT 520: Forecasting and Time Series
Modeling the Conditional Mean and the Conditional
Variance
I In the previous example, we just assumed the conditional
mean was zero.
I For some data, the mean process is not constant, and we can
combine a regression model or ARMA model for the mean
with an ARCH model for the errors.
I For example, an AR(1) model (with an intercept) for the
mean process, with ARCH(1) errors, would be

Yt = θ0 + φYt−1 + Zt ,

where Zt follows an ARCH(1) model.

I This model can be fit with the garchFit function in the
fGarch package.
I See the R example on the U.S. GNP data.
Hitchcock STAT 520: Forecasting and Time Series
The GARCH Model

I The ARCH model can be extended to the generalized

autoregressive conditional heteroscedasticity, or GARCH,
model.
I The GARCH model introduces one or more lags of the
conditional variance into the model.
I The GARCH(1, 1) model can be expressed as:

rt = σt|t−1 t
2 2 2
σt|t−1 = ω + βσt−1|t−2 + αrt−1

Hitchcock STAT 520: Forecasting and Time Series

The GARCH(p, q) Model

I This model can be generalized to the GARCH(p, q) model by

adding more lags of the conditional variance and/or squared
returns:

rt = σt|t−1 t
2 2 2
σt|t−1 = ω + β1 σt−1|t−2 + · · · + βp σt−p|t−p−1 +
2 2
α1 rt−1 + · · · + αq rt−q

I In this notation and in the garch function in the tseries

package, the first subscript is the number of GARCH terms
(number of β’s) and the second subscript is the number of
ARCH terms (number of α’s).
I In some books and software (like the garchFit function in the
fGarch package), the ordering of the subscripts is reversed.

Hitchcock STAT 520: Forecasting and Time Series

Specifying and Estimating a GARCH Model
I If the returns follow a GARCH(1, 1) model, then the squared
returns rt2 behave like an ARMA(1, 1) process.
I So if our model specification tools indicate the squared
returns are ARMA(1, 1), we can use a GARCH(1, 1) model for
the return process {rt }.
I In general, if {rt } is GARCH(p, q), then {rt2 } follows an
ARMA model with orders max(p, q) and p.
I Note that if q < p, then q is impossible to identify using this
rule, so the typical approach in that case is to fit a
GARCH(p, p) model and remove the terms whose estimated
coefficients are not significant.
I In GARCH models, the β’s and α’s are usually constrained to
be nonnegative, which is a sufficient (though not necessary)
condition for the conditional variances to be nonnegative.
I Parameter estimation is carried out through ML.
Hitchcock STAT 520: Forecasting and Time Series
Example on a Simulated GARCH(1, 1) Time Series
I See the R example for a simulated GARCH(1, 1) series with
ω = 0.02, α = 0.05, and β = 0.9.
I The ACF and PACF of this series show little autocorrelation
(just a bit at lags 3 and 20).
I But the ACFs and PACFs of the absolute values and squared
values show lots of autocorrelations (not resembling white
noise).
I The sample EACF of the squared values is unclear, possibly
suggesting ARMA(2, 2), which would imply a GARCH(2, 2)
for the original series.
I Alternatively, we can examine the sample EACF of the
absolute values, which suggests an ARMA(1, 1), which would
imply a GARCH(1, 1) for the original series.
I Fitting both models, the GARCH(1, 1) looks better, based on
the AICs and on the significance of the estimated coefficients.
Hitchcock STAT 520: Forecasting and Time Series
Example: The CREF Time Series

I If we fit a GARCH model to the CREF return series, we must

decide on the appropriate orders.
I The EACF of the series of squared returns indicates an
ARMA(1, 1) model, which implies a GARCH(1, 1) model for
the CREF returns.
I The EACF of the absolute returns is a bit unclear, but
possibly suggests the same model.
I The GARCH(1,1) model fit yields the estimates ω̂ = 0.01633,
α̂ = 0.04414, β̂ = 0.91704.
I Note that this model’s estimate of the long-term variance is
ω̂/(1 − α̂ − β̂) = 0.01633/(1 − 0.04414 − 0.91704) = 0.4206,
which is very close to the sample variance of 0.4161.

Hitchcock STAT 520: Forecasting and Time Series

Diagnostics: The CREF Time Series
I The usual diagnostics can be used to check the fit of the
GARCH model.
I The Jarque-Bera test and Ljung-Box test on the residuals
both indicate no problems.
I The Q-Q plot of the residuals shows approximate normality,
and the residual time plot and ACF of the squared residuals
show that the squared residuals resemble white noise, which
indicates a good model.
I A general Box-type test of the autocorrelations of the squared
residuals indicates no problems (large P-value).
I A plot of the ACF of the absolute residuals and the gBox test
on the absolute residuals shows some possible lag-2
autocorrelation.
I But overfitting with a GARCH(1, 2) model produces worse
results, and the AIC of the GARCH(1, 1) model is better, so
we conclude the GARCH(1, 1) model gives a good fit.
Hitchcock STAT 520: Forecasting and Time Series
An AR(1)-GARCH(1,1) Model Example

I The Dow Jones Industrial Average from April 20, 2006 to April
20, 2016 exhibits some autocorrelation in the original series of
returns, and lots of autocorrelation in the squared returns.
I We can fit an AR(1) model for the mean process, and a
GARCH(1, 1) model for the conditional variance, using the
garchFit function in the fGarch package.
I The plot of the predicted conditional standard deviation over
time shows lots of volatility during the time of the financial
crisis of 2008.

Hitchcock STAT 520: Forecasting and Time Series