Time series analysis

By
PRIYANGA V
2019604005
 Time series

 A Time Series is a set of statistical observations arranged

in chronological order. (or) Time series consists of
statistical data which are collected, recorded or observed
over successive increments.
Mathematically, Time Series is denoted as Yt
Y  Values of Variables ( Price, Temperature,... )
t Time(Hourly, daily, monthly, yearly,… )
 Time series is dynamic - change over time
 Components of Time series
Time series can be decomposed into 4 components

• Secular Trend (T)

• Seasonal variation (S)

• Cyclical variation (C)

• Irregular variation (I) or random movement

Considering the effects of these 4 components, two different models used in time series :
 Multiplicative model
Y(t) = T(t)× S(t)×C(t)× I (t)
Not necessarily independent but they can affect one another.
 Additive model
Y(t) = T(t) + S(t) + C(t) + I (t)
Independent of each other
 Secular trend
• The general tendency of a time series to increase, decrease or
stagnate over a long period of time is termed as Secular Trend or
simply Trend.

• Thus, it can be said that trend is a long term movement in a time

series (least 10-15 years data).

• Mathematically trend may be Linear or non-linear(curvi-linear)

Cont…

Upward trend: Population growth, agricultural production

Downward trend: Declining birth or death rate
 Seasonal variation

• Seasonal variations in a time series are fluctuations within a year

during the season.

• The important factors causing seasonal variations are: climate and

weather conditions, customs, traditional habits, etc.

Examples:

• Rainfall data recorded over months or seasons

 Cyclical variation
• The cyclical variation in a time series describes the medium-term
changes in the series, caused by circumstances, which repeat in cycles.

• Refer to swings (upswings and downswings) around a trend line.

• The duration of a cycle extends over longer period of time, usually

two or more years.

• Most of the economic and financial time series show some kind of
cyclical variation.
Cont…
 Irregular or random variations

• Irregular or random variations in a time series are caused by

unpredictable influences, which are not regular and also do not
repeat in a particular pattern.

• These variations are caused by incidences such as war, strike,

earthquake, flood, revolution, etc.

• There is no defined statistical technique for measuring random

fluctuations in a time series.
 Measurement of trend

• Graphic method

• Method of Moving average

• Method of Semi-Averages

• Method of Curve fitting by principles of least squares

 Graphic method

• Involves plotting the data in a two dimensional space.

• Provide the scatter diagram

• Based on the scatter diagram it is possible to observe the general

trend in time series.

• The scatter diagram generally used as a first step to decide the

period and nature of trend line, because this method by itself
does not provide an objection measurement of the trend
components.
Cont…
 Moving average

• Used to remove the cyclical components from the time series.

• Suitable, when the scatter diagram indicates that the trend is wavy.

• It provides the flexible trend line which alters its direction and rate of
change to confirm with bends in the trend.

4 steps involve in the computation of moving average

• Deciding the logical time period

• Calculation of moving totals

• Calculation of moving average

• Centering the moving average

Cont…
Cont…
Cont…

• The moving average curve provide the better idea of the trend in
price but irregular components is still present in the moving
average series.

• Moving average represents both the trend and irregular

components and not the trend alone.
 Semi- average straight line method

• This is another simple method to ascertain the nature of

trend in annual time series.

• Variables are divided into two equal parts

• If the number of observation is odd, the middle observation

is ignored.
Cont…
Cont…

Average of each part

Particulars 1 to 11 13 to 23

Sum of index 1434.69 2906.62

numbers
Average of index 130.43 264.24
numbers

Averages are plotted on a graph and joined by

the straight line (Trend line)

X axis Y axis
6 130.43
18 264.24
Cont…
Cont…

Equation of the straight line

where, = value of time series variable

= serial number allotted to the year t

a and b intercept and slope

= average of index number for second sub period (264.24)

= average of index number for first sub-period (130.43)

= serial number assigned to mid- year of second sub-period (18)

= serial number assigned to mid- year of first sub- period (6)

Cont…

This method is estimating the trend line, though simple, is not

recommended because the more objective method of ordinary
least square (OLS) is available to estimate the straight line
 Least square trend line method

This method of ascertaining the trend in a annual time

series involves estimating the coefficients a and b in the
following linear functional form

• value of time series variable

• = serial numbers assigned to the tthyear

• = random disturbance term

 Measurement of seasonal variation

• Seasonal average method

• Link relative method

• Ratio to trend method

• Ratio to moving average method

Seasonal average method
Cont…

Calculation of quarterly seasonal indices

Year I II III IV Total

1989 - - 127 134
1990 130 122 122 132
1991 120 120 118 128
1992 126 116 121 130
1993 127 118 - -
Total 503 476 488 524
Average 125.75 119 122 131 497.75
Quarterly 101.05 95.6 98.04 105.03
seasonal
indices
 Link relative method

• Pearson’s method

• Most widely used method for obtaining a seasonal index

• Less satisfactory than the moving average method because it

is not readily adaptable to the determination of changing
seasonal pattern.
Cont…

Steps involved:

• Calculate the average of link relative for each season.

• Convert the set of mean link relatives into the set of chain
relatives.

• Adjust the chain relatives for trend by using the adjustment factor

• Correct the adjusted chain relatives by using the correction factor.

 Ratio to trend method

• Based on multiple model of time series.

• Calculate the trend value for various time duration (monthly or

quarterly) with the help of Least square method.

• Express all original data as the percentage of trend on the basis of

the following formula.

=
Cont…

• The irregular components are removed in the process of

averaging each month’s ratio over the years and also through
correction factor.

• But the cyclical component is not completely eliminated

from the seasonal indices.

• Disadvantages- ignores the disturbing effect of cyclical

components.
 Ratio to moving average method

• Original time series are expressed as the percentage of moving

average instead of percentage of trend.

• Calculate the 12 monthly or 4 quarterly moving average (T*C)

Formula for calculating the moving average ratio

• Find the averages of seasonal variation for each month.

• These averages are to be adjusted by using the correction factor.

 Measurement of trend, cyclical and irregular
component simultaneously

• Additive hypothesis – Trend (AT) method

• Additive hypothesis- Trend and Moving Average (ATMA)

method

• Multiplicative hypothesis – Trend (MT) method

• Multiplicative hypothesis – Trend and Moving average

(MTMA) method
 Additive hypothesis – Trend (AT) method

• Three year weighted moving average of C+I components

represent cyclical components (C)

• The simple difference of C+I and C components leaves only

Irregular components
Additive hypothesis- Trend and Moving
Average (ATMA) method

Moving average of

Estimated linear Trend value

 Multiplicative hypothesis – Trend (MT)
method

• Cyclical (C) components are obtained by taking weighted three

years moving average (1:2:1) of indices CI components.

• Irregular components are obtained by dividing the indices of

cyclical components and multiplied by 100
 Multiplicative hypothesis – Trend and
Moving average (MTMA) method

• Cyclical (C) components are obtained by dividing the

original time series variable by moving average of original
time series .

• Irregular components (I) are obtained by dividing the

moving average of original time series by estimating linear
trend values ()
 Time series analysis and forecasting

The procedure of fitting a time series to a proper model is

termed as Time Series Analysis.

Purpose
Understand the historical pattern

Judge upon the current status

Make forecast for the future development

• To making a good forecast, fitting an adequate model to a time

series is important.
 Concept of Stationarity
• If time series data is said to be stationary, it’s mean and variance
remain same over the time.

• It is a necessary condition for building a time series model that is

useful for future forecasting.

• Two types of stationary processes

1. Strongly Stationary or Strictly Stationary

2. Weakly Stationary

• If the time series is non stationary, the solution to transform them into
stationary series by taking the difference of such time series variable.
 Methods to test the existence of unit
root/non-stationarity

• Graphical analysis

• Autocorrelation Function (ACF) and Correlogram

• Dicky- Fuller test

• Augmented Dicky Fuller (ADF)

• Philips- Perron (PP) unit root test

• Kwiatkowski–Phillips–Schmidt–Shin (KPSS)
Graphical analysis
0.00

-0.25

Correlogram
0 50 100 150 200 250 300 350 400 450 500
1.0
ACF-whitenoise

0.5
12.5
randomwalk
Stationary process
10.0
correlogram dies out
0.0 7.5
rapidly
5.0
-0.5
2.5

0 5 0.0
10
0 50 100 150 200 250 300 350 400 450 500
1.00
ACF-randomwalk

Non- stationary- 0.75

correlogram does not 0.50

die out
0.25

0 5 10
 Augmented Dicky Fuller (ADF)

Null hypothesis: (i.e., unit root or non-stationary)

Alternate hypothesis: (i.e., stationary)

The model can be expressed as

Where, = Actual time series value

t is the time or trend variable.

Cont…

• If the computed absolute value of tau statistic (|τ|) exceeds

the absolute critical value or Mac Kinnon Critical tau value,
we reject the hypothesis of , which means that the variable is
stationary.

• On the other hand, if the computed tau statistic (|τ|) value

does not exceed the absolute critical value, we do not reject
the null hypothesis, in which case the time series is non
stationary.
 Phillips and Perron (PP) Test

• Non-parametric statistical methods

• Serial correlation in the error terms without adding lagged

difference terms.

• ADF and PP test are same based on asymptotic analysis

• Better for small sample properties

H0 : Non stationary

H1 : Stationary
 Kwiatkowski–Phillips–Schmidt–Shin (KPSS)

• Test is based on linear regression.

• It breaks up a series into three parts: a deterministic trend (βt),

a random walk (rt), and a stationary error (εt)

Yt = rt + βt + εt

• Null hypothesis  Stationary

• Alternate hypothesis  Non stationary

 Different time series models
Linear time series models Non- linear time series models

Autoregressive (AR) Autoregressive Conditional

Heteroskedasticity (ARCH)

Moving Average (MA) Generalized ARCH (GARCH

Autoregressive Moving Average Exponential Generalized ARCH
(ARMA) (EGARCH)

Autoregressive Integrated Moving Threshold Autoregressive (TAR)

Average (ARIMA)

Autoregressive Fractionally Integrated Non-linear Autoregressive (NAR)

Moving Average (ARFIMA)

Seasonal Autoregressive Integrated Non-linear Moving Average (NMA)

Moving Average (SARIMA)
Methods of forecasting – Naïve models
– MA, AR, ARMA – ARIMA Box-
Jenkins approach
Autoregressive (AR) and Moving Average (MA)

• In an AR(p) model, the future value of a variable is assumed to be a

linear combination of p past observations and a random error together
with a constant term.

• In MA(q) model, a linear regression of the current observation of the

time series against the random shocks of one or more prior
observations.

• The random shocks are assumed to be a white noise process, i.e. a

sequence of independent and identically distributed (i.i.d) random
variables with zero mean and a constant variance.
 Autoregressive Moving Average
(ARMA)
• Autoregressive (AR) and moving average (MA) models can be
effectively combined together to form a general and useful class
of time series models, known as the ARMA models.

• Mathematically, an ARMA(p, q) model is represented as

• The constant p and q are known as the order of the number of

autoregressive terms and moving average terms.
 Autocorrelation and Partial Autocorrelation
Functions (ACF and PACF)
• For modeling and forecasting purpose, it is often useful to plot the ACF
and PACF against consecutive time lags.

• These plots help in determining the order of AR and MA terms.

• The ACF at lags k, denoted by ρk=

• If we plot ρkagainst k, the graph we obtain is known as the population

correlogram. It lies between -1 to +1.

• Similarly, the partial autocorrelation ρkkmeasures the correlation between

the observation that are k time periods after controlling for correlations
at intermediate lags (i.e., lags less than k).
 Theoretical pattern of ACF and PACF

Model Pattern of ACF Pattern of PACF

AR(p) Spikes decay Significant spikes through
exponentially lags p

MA (q) Significant spikes through Spikes decay

lags q exponentially

ARMA (p, q) Spikes decay Spikes decay

exponentially exponentially
 ARIMA (Auto Regressive Integrated
Moving Average) model

• Time series data usually contain unit root/ it is non-

stationary in nature that could be spurious.

• ARMA models are inadequate to properly describe non-

stationary time series, which are frequently encountered in
practice.

• In ARIMA models, a non-stationary time series is made

stationary by applying finite differencing of the data points.
 ARIMA (p, d, q)
• Here, p, d and q are integers greater than or equal to zero and refer
to the order of the autoregressive, integrated, and moving average
parts of the model respectively.

• The integer d controls the level of differencing. Generally d=1 is

enough in most cases.

• d=0 - ARMA(p,q) model.

• ARIMA(p,0,0)- AR(p) model

• ARIMA(0,0,q) - MA(q) model.

• ARIMA(0,1,0) - Random Walk model

 Box and Jenkins methodology

• Identification: Identification means finding out the

appropriate values of p, d and q.

• Estimation: After identifying the appropriate ARIMA (p, d,

q) model, the next stage is to estimate the parameters of
autoregressive and moving average terms included in the
equation.
• Diagnostic checking: In order to examine whether the chosen
model fits the data reasonably well, one simple diagnostic is to
obtain the residuals and obtain Autocorrelation (ACF) function
and Partial Autocorrelation (PACF) function of these residuals. If
the residuals estimated from this model are white noise, we can
accept the model.

• Forecasting: The forecasts obtained from this method are more

reliable than those obtained from the traditional econometric
modelling.
 Model Selection Criteria

• Helps to find appropriate model for forecasting.

For ARIMA model,

• Akaike Information Criteria

• Schwarz Information Criterion

Where, T denotes the number of observation

denotes the mean square error.

• The ARIMA model with lowest AIC/ SIC value will be more
appropriate for forecasting.
 Seasonal Autoregressive Integrated Moving
Average (SARIMA) Models

• Box and Jenkins have generalized this model to deal with

seasonality. (SARIMA model)
• In this model seasonal differencing of appropriate order is used to
remove non-stationarity from the series.
• A first order seasonal difference is the difference between an
observation and the corresponding observation from the previous
year and is calculated as
• For monthly time series s = 12 and for quarterly time series s = 4.
• This model is generally termed as the
SARIMA (p, d, q) (P, D, Q)s
Volatility estimation – Conditional
heteroscedasticity – ARCH models –
GARCH models
 Volatility analysis
• Volatility poses a big challenge in modeling financial time series
like agricultural prices, stock prices, exchange rates, inflation, etc.
which exhibits the phenomenon of volatility clustering.

• Variation in the economic variables over a period of time is

referred as volatility.

• Opposite to homoscedasticity, data in which the expected value of

the error terms is not equal, is said to be heteroscedasticity.

• Such a time series with varying variance are modeled by ARCH

and GARCH models.
Cont…

• ARCH (Autoregressive Conditionally Heteroskedasticity) model

was introduced by Engle (1982) and GARCH (Generalized
ARCH) by Bollerslev (1986).

• These ARCH and GARCH models allowed the conditional

variance to change over time as a function of past errors leaving
the unconditional variance constant; it was the first model that
provides a systematic form for volatility modeling.
 ARCH- LM test

• The most commonly used method to test for ARCH effect is

Lagrange-Multiplier test.

• In this test, first estimate the mean equation using OLS,

• The presence of ARCH effect or conditional heteroscedasticity

(whether or not volatility varies over time) has to be tested
through the squared residuals of the series.
Cont…

Lagrange’s Multiplier (LM) test

• H0: γ1 = 0 and γ2 = 0 and γ3 = 0 and ... and γq = 0 in the linear

regression where squared residuals were regressed them on q own
lags to test for ARCH of order q

• Test statistic is defined as n.R2 and is distributed as a χ2 with q

degrees of freedom
 ARCH Model
• ARCH models are defined in terms of the distribution of errors of
a linear regression model by taking dependent variable as time
series which is considered as an autoregressive process.

• Engle (1982) developed a model for based on past squared errors

(ut) of above equation known as conditional variance.
Cont…

• The conditional variance depends on the previous q lagged

innovations is denoted as ARCH (q) and is represented as

• In the equation it can be seen that large values of the innovation of

time series has a bigger impact on the conditional variance because
they are squared, which means that a large shock have tendency to
follow the other large shock and this behaviour is named as the
volatility clustering.
 The GARCH Specification

• The GARCH model had two distinct specifications-one for

the conditional mean and one for the conditional variance
are provided and the standard GARCH (1,1) specification
was presented below:
 Cont…

1. The mean equation was a function of exogenous variables with an

error term.

2. The conditional variance equation was a function of three terms

namely:
• The mean :ω

• Volatility from the previous period (measured as the lag of the

squared residual from the mean equation): (the ARCH term)
• Last period's forecast variance: (the GARCH term)
 Cont…

• An ordinary ARCH model was a special case of a GARCH

specification in which there were no lagged forecast variances in
the conditional variance equation.

• The resultant sum of theses coefficients () are estimated.

• The sum of coefficients very close to 1 would indicate the volatility

shocks are quite persistent in the series.
Others…
Volatility models Proposed by

Exponential GARCH Nelson (1991)

(EGARCH) model
Conditional heteroscedastic Tsay (1987)
autoregressive moving average
(CHARMA) model
Random coefficient Nicholls and Quinn (1982)
autoregressive (RCA) model
Stochastic volatility (SV) model Melino and Turnbull (1990),
Harvey, Ruiz, and Shephard (1994),
& Jacquier, Polson, and Rossi (1994)
 Time Series Forecasting Using Artificial
Neural Networks

• It has been suggested as an alternative technique to time series

forecasting.

• The basic objective was to construct a model for mimicking the

intelligence of human brain into machine.

• Similar to the work of a human brain, ANNs try to recognize

regularities and patterns in the input data, learn from experience
and then provide generalized results based on their known
previous knowledge.
Cont…

Salient features of ANNs

• Data-driven and self-adaptive in nature

• Inherently non-linear, which makes them more practical and

accurate in modeling complex data patterns.

• Universal functional approximators

Cont…

The ANN Architecture

• The most widely used ANNs in forecasting problems are multi-

layer perceptrons (MLPs), which use a single hidden layer feed
forward network (FNN).

• The model is characterized by a network of three layers, viz.

input, hidden and output layer, connected by acyclic links. There
may be more than one hidden layer.

• The nodes in various layers are also known as processing

elements.
Cont…

Three-layer feed forward architecture of

ANN models
 Co-integration
Stationarity among non-stationary variables

If two or more series are themselves non stationary, but a

linear combination of them is stationary, then they are said to be
co-integrated

In economics
Two variables are said co-integrated when
they have long-term or equilibrium relationship
between them
 Testing for Co-integration

Testing for co-integration is the necessary step to check

whether the empirical modelling has meaningful relationships or
not.

1. Engle–Granger test

2. Phillips–Ouliaris test

3. Johansen’s test
 Engle-Granger test

• If Yt and Xt are two spatially separated time series, if they possess the
stochastic trends and are integrated of the same order, say I (d), there
exist a co-integrating relationship. It can be expressed as below,

Yt = α + β Xt + ut

Where, ut is the error term

β is the co-integrating coefficient

• Yt and Xt are cointegrated only if the residual term ut is integrated of

order one I(1).
 Phillips-Ouliaris methods

• Residual-based unit root test

• Two tests
 Variance ratio test
 Multivariate trace statistic
• The tests is also invariant to normalization of the co-integration
relationship (i.e. Whatever variable is taken to be the dependent
variable, the test will yield the same results).
H0 : No co-integration exists

H1 : Co-integration exists
Cont…

• They are implemented on matrices or multivariate series and are

both based on residuals of the first order vector auto regression
equation
Zt= Π zt-t + εt

• Where Zt is partitioned as = (yt , xt) with a dimension of xt equal to

(m = n + 1), Π is a regression coefficient, and n is equal to the
number of variables.
 Johansen Maximum Likelihood (ML) test

• Multivariate Approach to Co-integration- More than 2 variables in the

model

• Based on relationship between the rank of a matrix and its characteristic

roots

• Derived the MLE using sequential tests for determining the number of co-
integrating vectors

• Test statistics are based on

• Trace value

• Maximum Eigen value

Cont…
• In testing for efficiency of two spatially separated markets (which
is the necessary condition for market integration).

• Rank of the coefficient matrix at most ‘r’ ( r = 0, 1, 2,3…n-1)

ΔYt= α + βt + (p – 1) Yt–1 + θ1ΔYt–1 + . . . + θk–1ΔYt–k+1 + Wt

λtrace= –T

λmax= T In (1 – λr–1)

H0 : No co-integration (r=0)

H1 : Co-integration exist (r=1)

 Error Correction Mechanism (ECM).

• Residuals from the equilibrium regression (Engle- granger test or

Johansen co-integration test) can be used to estimate the error
correction mechanism (ECM).

• To analyse the long term and short term effects of the variables
as well as to find the speed of adjustment of disequilibrium to
the original equilibrium condition.

• This coefficient is the lagged residual terms of the long run

relationship.
Cont…

Cointegration regression model

Yt = α + βXt + ut

Save residuals

ΔYt = C + β Δ Xt - δ ut-1

δ  speed of error correction which is expected to be

significant and negative
 Granger causality test

• To know the direction of long run causal relation between the series.

• Co-integration is present  Test analyse the direction of this co-

movement relationship.

• Tests come in pairs to testing whether variable Xt causes variable Yt

and vice versa.

• All permutations are possible viz., univariate Granger causality from

Xt to Yt or from Yt to Xt, bivariate causality or absence of causality.
 Cont…

• An autoregressive distributed lag (ADL) model for the

Granger-causality test was specified as below:

• Where, t is the time period,

• u1t and u2t are the error terms,

• X and Y are the prices series of different markets.