Module 2 Part 1 – Types of

Forecasting Models Quantitative Forecasting

and Simple Linear Causal Models

Regression  are quantitative forecasting models wherein the

variable to be forecast is influenced by or
Analysis correlated with other variables included in the
model.
 include regression models and other more
Types of Forecasting Models (Render et al., 2016) complex models
Qualitative Models  example: daily sales of bottled water might
depend on the average temperature, the average
 based on judgmental or subjective factors (lacks humidity, and so on.
historical data, relies on expert opinion and  utilizes variables that are related to each other
individual experiences)  regression models have linear relationship
 example: introducing a new product (forecasting  in simple linear regression, one dependent and
demand is difficult due to lack of any historical one independent variable only.
sales data)
Time-Series Models
 are also quantitative forecasting models/
Qualitative Forecasting techniques that attempt to predict the future
Delphi method values of a variable by using only historical data
on that one variable.
 this iterative group process allows experts in  these models are extrapolations of past values of
different places to make forecasts; involves three that series
different participants  example: using the past weekly sales for lawn
 decision makers – usually consists of 5 to 10 mowers in making the forecast for future sales
experts
 staff personnel – assist the decision makers
 respondents – group of people whose judgement
are valued and being sought
Jury of executive opinion
 a method which takes the opinions of a small
group of high-level managers, often in
combination with statistical models, and results in
a group estimate of demand.
 one small group
Regression Analysis (Render et al., 2016)
Sales force composite
A forecasting technique with generally two purposes:
 an approach where each salesperson estimates
1. to understand the relationship between two
what sales will be in his or her region; these
variables.
forecasts are reviewed and combined at the
2. to predict the value of one based on the other.
district and national levels to reach an overall
forecast. Its’ applicability is virtually limitless (e.g., level of
 bottom-up approach education and income, price of a house and the
square footage, advertising expenditures and sales,
Consumer market survey
etc.).
 a method that solicits input from customers or
The variable predicted is called the dependent
potential customers about their future purchasing
variable or response variable. The value of this
plans; helps not only in preparing a forecast but
response variable is said to be dependent upon the
also in improving product design and planning for
value of an independent variable (explanatory
new products
variable or predictor variable).
 “grass roots approach” because they reach out to
actual consumers
Correlation Analysis The module will focus on the simple linear model
commonly referred to as a simple regression
The analysis of bivariate data (two sets of data) equation.
typically begins with a scatter plot that displays each
observed pair of data (x, y) as a dot on the x-y plane. Types of Relationships:
Correlation analysis is used to measure the strength of
the linear relationship between two variables.
Correlation is only concerned with strength of the
relationship.
No causal effect is implied with correlation.
The sample correlation coefficient (like Pearson’s r
and Spearman rho) measures the degree of linearity
in the relationship between two random variables X
and Y, with values in the interval [-1, 1].

Scatter plots showing various correlation coefficient

values
In Excel, use these functions to get the value of the
correlation coefficient. Simple Linear Regression Model

 =CORREL(array1, array2) A statistical technique for analyzing and summarizing

 =PEARSON(array1, array2) relationships between two continuous quantitative
variables.
Has only independent, predictor, or explanatory (X)
Test for significant correlation using Student’s t: variable.
The sample correlation coefficient r is an estimate of The relationship between X and Y is described by a
the population correlation coefficient ρ (Greek linear function.
alphabet rho).
Changes in dependent, response, or outcome (Y)
There are no flat rules for a “high” correlation because variable can be attributed to changes in X.
sample size must be taken into consideration.
Before you can proceed to simple linear regression,
To test the hypothesis: H 0 : ρ=0, the test statistic is you need to analyze first if there is a relationship using
the correlation analysis.

√
t computed = r
n−2
1−r 2
The Population Regression Model
After calculating this value, we can find its p-value by
using Excel’s function = T . DIST .2 T (t , deg ⁡freedom )

Regression Analysis
The hypothesized relationship may be linear,
quadratic or some other form.
The next slide presents some of the possible patterns.
Simple Linear Regression Equation meaningful because no apartment can have 0 square
foot.

Equation: MPG=49.22−0.079 Horsepower

Interpretation: Each unit increase in engine
horsepower decreases the fuel efficiency by 0.079
mile per gallon. The intercept is not meaningful
The sign of your slope will determine the direction of
because a zero-horsepower engine does not exist.
the line, whether it is tilting to the left or right.
The following formulas can be used to compute the
slope (b 1) and the intercept (b 0 ): Prediction Using Regression
One of the main uses of regression is to make
predictions. Once we have a fitted regression
equation that show the estimated relationship
between X and Y, we can plug in any value of X (within
the range of our sample x values) to obtain the
prediction for Y.
Equation: Sales=268+7.37 Ads
Prediction: If the firm spends $10 million on
advertising, its predicted sales would be $341.7
Interpreting an Estimated Regression Equation million; that is, Sales=268+7.37 ( 1 ) =341.7
The slope tells us how much, and in what direction,
the dependent or response variable will change for
each one unit increase in the predictor variable. Equation: DrugCost =410+550 Dependents
On the other hand, the intercept is meaningful only if Prediction: If an employee has four dependents, the
the predictor variable would reasonably have a value predicted annual drug cost would be $2,610; that is,
equal to zero. DrugCost =410+550 ( 4 )=2,610
Equation: Rent =150+1.05 SqFt
Examples: Prediction: The predicted rent on an 800-square foot
apartment is $990; that is,
Equation: Sales=268+7.37 Ads
Rent =150+1.05 ( 800 )=990
Interpretation: Each extra P1 million of advertising will
generate P7.37 million of sales on average. The firm
would average P268 million of sales with zero Equation: MPG=49.22−0.079 Horsepower
advertising. However, the intercept may not be
meaningful because Ads = 0 may be outside the range Prediction: If an engine has 200 horsepower, the
of observed data. predicted fuel efficiency is 33.42 mpg; that is,
MPG=49.22−0.079 ( 200 )=33.42

Equation: DrugCost =410+550 Dependents

Example 1: Triple A Construction Company renovates
Interpretation: Each extra dependent raises the mean old homes in Albany. Over time, the company has
annual prescription drug cost by $550. An employee found that its dollar volume of renovation work is
with zero dependents averages $410 in prescription dependent on the Albany area payroll. The figures for
drugs. Triple A’s revenues and the amount of money earned
by wage earners in Albany for the past 6 years are
presented in table below.
Equation: Rent =150+1.05 SqFt
Interpretation: Each extra square foot adds $1.05 to
monthly apartment rent. The intercept is not
 Normality of error – the error values are normally
distributed for any given value of X
Equal variance or homoskedasticity – the probability
distribution of the errors has constant variance.
Solution: To investigate the relationship between
Example 1 Excel output
variables, it is helpful to look at the scatter diagram or
scatter plot below.

For the scatterplot:

Note: The graph indicates that higher values for the
local payroll seem to result in higher sales for the 1. Highlight X array and Y array.
company. It is not a perfect relationship because not 2. Choose Insert.
all the points lie in a straight line, but there is a 3. Choose Scatter among the chart types available.
relationship. 4. Edit the axis labels.
Regression Calculations for Triple A Construction:

For regression:
Computing the slope and the intercept of the 1. Go to Data, choose Data Analysis.
regression equation, we have: 2. Choose Regression among the Data Analysis Tools.
3. Fill up necessary fields.
4. Click OK.

Note: Each time a payroll increases by $100 million

(represented by X), we expect the sales to increase by
$125,000 since b 1=1.25($ 100,000).

The estimated regression equation therefore is: The estimated regression equation is:
Y =2+ 1.25 X∨Sales=2+1.25(Payroll ) Sales=2+1.25∗(Payroll)

Assumptions of Regression (L.I.N.E)

Linearity – the relationship between X and Y is linear
Independence of errors – the error values (difference
between observed and estimated values) are
statistically independent.
Intercept is = 2
 Comparing Standard Errors
SYX is a measure of the variation observed Y values
from the regression line.

The magnitude of SYX should always be judged

Assessing Fit: Coefficient of determination, R2
relative to the size of the Y values in the sample data.
The coefficient of determination is the portion of the
total variation in the dependent variable that is
explained by the variation in the independent
variable.
It is also called r-squared and is obtained by:

2 SSR regression ∑ of squares

r= =
SST total ∑ of squares
Inferences about the slope using the t-test
2
0≤r ≤1
The t-test for a population slope is used to determine
if there is a linear relationship between X and Y.
Example 1 Coefficient of determination Null and alternative hypotheses
H0: β1 = 0 (no linear relationship)
H1: β1 ≠ 0 (linear relationship does exist)

Standard Error of Estimate

The standard deviation of the variation of Example 1 Inferences about the slope
observations around the regression line is estimated
by:

√
n

∑ (Y i−Y^ i )2
SYX =
√ SSE
n−2
= i=1
n−2
Where:
The estimated regression equation is:
SSE = error sum of squares
Sales=2+1.25∗(Payroll)
n = sample size
The slope of this model is 1.25. Is there a relationship
Example 1 Standard error of estimate between the payroll and the sales?
Example 1 Excel output

Example 1 Inference about slope

 2. Examine the Box-and-Whisker Plot of the
Residuals
3. Examine the Histogram of the Residuals
H0: β1 = 0 4. Construct a normal probability plot.
5. Construct a Q-Q plot.
H1: β1 ≠ 0
If residuals are normal, the probability plot and the Q-
Reject the null hypothesis since p < α. Q plot should be approximately linear.
There is sufficient data to conclude that there is a
statistically significant relationship between the
payroll and the sales.

Checking the assumptions by examining the residuals

Residual Analysis for Linearity: Plot X against residuals

What can we do when residuals are not normal?

Aside from visually examining the scatter plots of the 1. Consider trimming outliers – but only if they
IV and DV to assess linearity, the scatter plot of the IV clearly are mistakes.
versus the residuals may also be examined. The plots 2. Can you increase the sample size? If so, it will
at the left show curve patterns which indicates that help assure asymptotic normality of the
the data relationship is not linear. Another model estimates.
should be used. 3. You could try a logarithmic transformation of the
Residual Analysis for Equal Variance: Plot X against variables. However, this is a new model
residuals specification with a different interpretation of
coefficients.
4. You could do nothing, just be aware of the
problem.
Residual Analysis for Independence of Errors: Plot
times series X against residuals

Residual Analysis for Equal variance: Plot predicted

values against residuals

Independence of errors means that the distribution of

errors is random and is not influenced by or
correlated to the errors in prior observations.

Residual Analysis for Normality:

1. Examine the Stem-and-Leaf Display of the
Residuals
Clearly, independence can be checked when we know Start with a scatter plot of X on Y to observe possible
the order in which the observations were made. The relationship.
opposite of independence is auto-correlation.
Perform residual analysis to check the assumptions.
 Plot the residuals vs X to check for violations of
Measuring Autocorrelation assumptions such as equal variance.
 Use a histogram, stem and leaf display, box and
Another way of checking for independence of errors is whisker plot or normal probability plot of the
by testing the significance of the Durbin Watson residuals to uncover possible non-normality.
Statistic.
If there is any violation of any assumption, use
The Durbin-Watson Statistic measures detects the alternative methods or models.
presence of autocorrelation.
If there is no evidence of assumption violation, then
It is used when data are collected over time to detect test for the significance of the regression coefficients.
the presence of autocorrelation.
Avoid making predictions or forecasts outside the
Autocorrelation exists if residuals in one time period relevant range.
are related to residuals in another period.
The presence of autocorrelation of errors (or
residuals) violates the regression assumption that
residuals are statistically independent.
Module 2 Part 2 – Multiple
Regression Analysis

Multiple Regression
Multiple regression extends simple regression to
include several independent variables (called
Example 1 Excel output for assessing assumptions predictors or explanatory variables).

It is required when a single-predictor model is

inadequate to describe the relationship between the
response variable (Y) and its potential predictors
(X 1 , X 2 , X 3 , …).
The interpretation is similar to simple regression since
simple regression is a special case of multiple
regression.

Calculations are done by computer.

Using multiple predictors is more than a matter of

“improving its fit”. Rather, it is a question of specifying
a correct model.

A low R2 in a simple regression model does not

necessarily mean that X and Y are unrelated but may
simply indicate that the model is incorrectly specified.
The residual plot shows that the assumptions of
linearity and constant variance are satisfied. Omission of relevant predictors (model
misspecification) can cause biased estimates and
The assumption of normality of residuals is satisfied misleading results.
since the points follow a straight line.

Limitations of Simple Regression

Strategies when performing regression analysis
Multiple relationships usually exist. Fitted Regression: Comparison Between a 1-Predictor
Model Versus a 2-Predictor Model
The estimates are biased if relevant predictors are
omitted.

The lack of fit (low R-squared) does not show that X is

unrelated to Y if the true model is multivariate.

Simple regression is only used then there is a

compelling need for a simple model, or when other
predictors have only modest effects and a simple
logical predictor “stands out” as doing a very good job
Example 1
all by itself.
A distributor of frozen dessert pies wants to evaluate
factors though to influence demand.
The Population Regression Model
 the dependent variable is pie sales (units per
Y = β0 + β 1 X 1 + β 2 X 2 +…+ β k X k + ε week)
 the independent variables are price (in USD) and
Where: advertising cost (in hundred USD)
 Y is the response variable. The data are collected for 15 weeks.
 β 0 , β 1 , β 2 , … , β k are the unknown regression
coefficients.
 X 1 , X 2 , … , X k are the predictor variables.
 ε is the random error term.
In the population regression model, the random error
ε represents everything that is not part of the model

The unknown regression coefficients, denoted by

Greek letters, are parameters

Each coefficient shows the change in the expected

value of y for a unit change in Xi while holding
everything constant (ceteris paribus).

The errors are assumed to be unobservable,

independent random disturbances that are normally
distributed with zero mean and constant variance.
Under these assumptions, the ordinary least squares
(OLS) estimation method yields unbiased, consistent,
efficient estimates of the unknown parameters.
Interpretation of The Regression Coefficients

Sales=306.516−24.975 ( X 1 ) +74.131 ( X 2 )
The Estimated Regression Equation
b 1=−24.975
Y^ =b0 +b 1 X 1 +b 2 X 2+ …+b X k
Sales will decrease, on average by 24.975 pies per
Where: week for each $1 increase in selling price, net of the
effects of changes due to advertising.
 Y^ is the predicted value of the response variable.
 b 0 is the estimated intercept or constant. b 0=74.131
 b 1 , b2 , bk are the estimated slope coefficients.
Sales will increase, on average, by 74.131 pies per
 X 1 , X 2 , … , X k are the predictor variables.
week for each $100 increase in advertising cost, net of
the effects of changes dues to price.
Predict sales for a week if the selling price is 6.50 and
the advertising cost is $420:

Sales=306.516−24.975 ( 6.50 ) +74.131 ( 4.20 ) =780.2137

Note that advertising it in $100s, so $420 means that
X 2 =4.20 .

Assessing Overall Fit: Coefficient of Multiple

Assessing Overall Fit: Residuals Determination

Similar to simple regression, there is one residual for The coefficient of multiple determination reports the
every observation in a multiple regression: proportion total variation in Y that is explained by the
variation of all predictor variables taken together.
e i= y i− ^yi for i=1 ,2 , … , n
It is also called r-squared and is obtained by:

2 SSR regression ∑ of squares

r= =
SST total ∑ of squares
2
0≤r ≤1

Assessing Overall Fit: F-Test for Significance

In simple regression, the F-test always yield the same

p-value as a two-tailed t-test for zero correlation. The Adjusted R-Squared
2
relationship between the test statistics is F calc=t calc . R-squared decreases when a new predictor variable X
is added to the model. This can be a disadvantage
Before determining which, if any, of the individual
when comparing models.
predictors are significant, we perform a global test for
overall fit using the F-test. What is the net effect of adding a new variable?

For a regression with k predictors, the hypotheses to  We lose a degree of freedom when a new variable
be tested are: is added.

H 0: All the true coefficients are zero Did the new X variable add enough independent
power to offset the loss of one degree of freedom?
H 1: At least one of the coefficients is nonzero.
The adjusted R2 shows the proportion of variation in Y
If: explained by all X variables adjusted for the number of
Significance F/P-value>alpha (.05), accept null X variables used.
hypothesis.

Significance F/P-value<alpha (.05), reject null

hypothesis.

It penalizes excessive use of unimportant predictor

variables.
It is smaller than R2.

It is useful when comparing models.

If we cannot reject the hypothesis that a coefficient is

zero, then the corresponding predictor does not
significantly contribute to the prediction of Y.

Determining How Many Predictors If a predictor is proven not relevant you can omit it
when writing your equation. Therefore, in some cases,
One way to prevent overfitting the model is to limit
the simple regression is a special case of multiple
the number of predictors based on the sample size.
regression. E.g., if you have 3 predictors and two of
Evans’ Rule (conservative): n /k ≥ 10 (at least 10 them are irrelevant.
observations per predictor)

Doane’s Rule (relaxed): n /k ≥ 5 (at least 5

Detecting Multicollinearity
observations per predictor)
When the predictor variables are related to each
These rules are merely suggestions.
other instead of being independent, we have a
condition known as multicollinearity.

Significance of Predictors Multicollinearity induces variance inflation and makes

the t statistics less reliable.
We are usually interested in testing each estimated
coefficient to see whether it is significantly different Least squares estimation fails when this condition is
from zero, that is, if a predictor variable helps explain present.
the variation in Y.
Ways of detecting multicollinearity:
Use t-tests of individual variable slopes.
 To check whether 2 predictors are correlated,
Shows if there is a linear relationship between the compute the correlation coefficients. Suspect
variables Y and X i . multicollinearity if two predictors are highly
correlated (r ≥ 0.80) or if the correlation
Hypotheses: coefficient exceeds the multiple R.
 Multicollinearity is present if variance inflationary
factor (VIF) is at least 10. The VIF is provided in
regression output in JASP.

Regression Diagnostics

Independence of errors – the error values (difference

between observed and estimated values) are
statistically independent OR non-autocorrelated. (for
time-series data and panel data)

Normality of error – the error values are normally

distributed for any given value of X

Equal variance or homoskedasticity – the probability

distribution of the errors has constant variance.
Checking the assumptions by examining the residuals

Residual Analysis for Equal variance:

Plot predicted values against residuals

Clearly, independence can be checked when we know

the order in which the observations were made. The
opposite of independence is auto-correlation.

Residual Analysis for Normality:

1. Examine the Stem-and-Leaf Display of the

Residuals
2. Examine the Box-and-Whisker Plot of the
Residuals
3. Examine the Histogram of the Residuals
4. Construct a normal probability plot.
Measuring Autocorrelation
5. Construct a Q-Q plot.
Another way of checking for independence of errors is
If residuals are normal, the probability plot and the Q-
by testing the significance of the Durbin Watson
Q plot should be approximately linear.
Statistic.

The Durbin-Watson Statistic measure detects the

presence of autocorrelation.

It is used when data are collected over time to detect

the presence of autocorrelation.

Autocorrelation exists if residuals in one time period

are related to residuals in another period.

The presence of autocorrelation of errors (or

residuals) violates the regression assumption that
residuals are statistically independent.

Residual Analysis for Independence of Errors:

Plot times series X against residuals

Independence of errors means that the distribution of

errors is random and is not influenced by or
correlated to the errors in prior observations.
Module 2 Part 3 – Time
Series Analysis and
Forecasting Accuracy
Forecasting methods can be classified as qualitative A time series plot is a graphical presentation of the
or quantitative. relationship between time and the time series
variable; time is represented on the horizontal axis
 Qualitative methods generally involve the use of
and values of the time series variable are shown on
expert judgment to develop forecasts. Such
the vertical axis.
methods are appropriate when historical data on
the variable being forecast are either unavailable
or not applicable.
Major Areas of Forecasting
 Quantitative forecasting methods can be used
when Demand Forecasting
(1) past information about the variable being
forecast is available,  Predicts timing and quantity of demand of firm’s
(2) the information can be quantified, and commodities
(3) it is reasonable to assume that the past is Economic Forecasting
prologue (i.e. the pattern of the past will
continue into the future).  Predicts future business conditions with reference
to the underlying economic factors such as
A forecast is simply a prediction of what will happen in inflation, GDP, GNP, etc.
the future. Managers must accept that regardless of
the technique used, they will not be able to develop Technology Forecasting
perfect forecasts.  Predicts possible technological advancements in
If the historical data are restricted to past values of the future such latest digital devices and systems,
the variable to be forecast, the forecasting procedure future software, and hardware, etc.
is called a time series method and the historical data
are referred to as a time series.
Forecasting Factors
The objective of time series analysis is to uncover a
pattern in the historical data or time series and then  Sales, Production
extrapolate the pattern into the future; the forecast is  Interest rates
based solely on past values of the variable and/or on  Funds
past forecast errors.  GNP
 Technological status
 Inventory
Time Series and Forecasting: Overview  Scheduling
 Facility layout
 Past – Historical Information  Workforce, and so on
 Present – Analysis of the Past information to
uncover design or pattern or model.
 Future – Forecasting or predicting possible Time Series
emerging event in the future based on the model
made at present based on the analyzed past Set of data depending on time
information.
A series of values over a period of time
A time series is a sequence of observations on a
Sequence of observations on a variable measured at a
variable measured at successive points in time or over
successive points in time or over successive period of
successive periods of time.
time
 May be taken every hour, every day, week,
Collection of magnitudes belonging to different time
month, or year, or at any other regular interval.
periods of some variable or composite of variables
 The pattern of the data is an important factor in
such as production of steel, per capita income, gross
understanding how the time series has behaved in
national income, price of tobacco, index of industrial
the past
production.
 If such behavior can be expected to continue in
the future, we can use it to guide us in selecting
an appropriate forecasting method.