Simulation and

Modeling
Lecture #4

Building Valid, Credible, and Appropriately Detailed Simulation

Models
By
Ihab Ali, Ph.D., SMIEEE
Professor of Networks and Security
Helwan University
“Edited from Original Slides by one of my Students”
 INTRODUCTION

The Target of this chapter is as follows :

• To Present a practical discussion of how to build valid and

credible models.
• To provide guidelines on how to determine the level of detail for
a model of a complex system
• To define the important terms used in this chapter, including
verification, validation, and credibility

• Note : Credible model is not necessarily valid, and vice versa.

 INTRODUCTION (Continued)

• Verification
is concerned with determining whether the “assumptions document” has
been
correctly translated into a computer “program,” i.e., debugging the
simulation computer program.

• Validation
is the process of determining whether a simulation model is an accurate
representation
of the system

• Credibility
if the manager and other key project personnel accept them as
“correct.” ,so,
a simulation model and its results have credibility
 INTRODUCTION (Continued)

• Timing and relationships of validation, verification,

and establishing credibility.
GUIDELINES FOR DETERMINING THE LEVEL OF MODEL DETAIL

• We must determine what aspects of a complex real-world system

actually need to be incorporated into the simulation model.

• It is rarely necessary to have a one-to-one correspondence

between each element of the system and each element of the
model.

• Modeling each aspect of the system will seldom be required to

make effective decisions and result in excessive model execution
time
GUIDELINES FOR DETERMINING THE LEVEL OF MODEL DETAIL
(Continued)

• Do not have more detail in the model than is necessary to address the issues of
interest.

• The level of model detail should be consistent with the type of data available

• In virtually all simulation studies, time and money constraints are a major factor
in
determining the amount of model detail.

• If the number of factors (aspects of interest) for the study is large, then use a
“coarse” simulation model or an analytic model.
 VERIFICATION OF SIMULATION COMPUTER PROGRAMS

• we will discuss eight techniques that can be used to debug the computer
program of a simulation model

• Technique 1

 The simulation model’s main program and a few of the key subprograms
should be written and debugged first,

 Next, additional subprograms or levels of detail should be added and debugged successively.

 it is better to start with a “moderately detailed” model, which

 is gradually made as complex as needed, than to develop “immediately”
 a complex model,

 This will lead to developing a model that satisfactorily represents the system under study
 VERIFICATION OF SIMULATION COMPUTER PROGRAMS
(Continued)

• Technique 2

 It is advisable in developing large simulation models to have more than

one person review the computer program

 In some organizations, this idea is implemented formally and is called a

structured walk-through of the program.

The subprograms’ developer goes through the programs but does not
proceed
from one statement to another until everyone in the modeling team is
convinced that a statement is correct.
 VERIFICATION OF SIMULATION COMPUTER PROGRAMS
(Continue..)

• Technique 3

• Run the simulation under a variety of settings of the

input parameters, and check to see that the output is
reasonable
 VERIFICATION OF SIMULATION COMPUTER PROGRAMS
(Continue..)
• Technique 4

One of the most powerful techniques that can be used to debug a DES
program is a “trace.”

In a trace, the state of the simulated system (the contents of the event list, the
state variables, certain statistical counters, etc.,) are
displayed just after each event occurs and are compared with hand
calculations to see if the program is operating as intended.

 It is preferable to use an interactive debugger to find programming errors,

An interactive debugger allows an analyst to stop the simulation at a selected
point in time, and to examine and change the values of certain variables
 VERIFICATION OF SIMULATION COMPUTER PROGRAMS
(Continued)
Partial trace for the single-server queue

Theoretical values (T) and simulation estimates (S) for a

simplified
job-shop model (M/ E2/1 queue)
 VERIFICATION OF SIMULATION COMPUTER PROGRAMS
(Continue..)

• Technique 5

The model should be run, when possible, under

simplifying assumptions for which its
true characteristics are known or can easily be computed.

• Theoretical values (T) and simulation estimates (S) for

a simplified job-shop model (four multiserver queues in series)
 VERIFICATION OF SIMULATION COMPUTER PROGRAMS
(Continue..)

• Technique 6
With some types of simulation models, it may be helpful to
observe an animation of the simulation output

• Technique 7
Compute the sample mean and sample variance for each
simulation input probability distribution, and compare them
with the desired (e.g., historical) mean and variance.
This suggests that values are being correctly generated from
these
distributions (gamma and Weibull distributions)
 VERIFICATION OF SIMULATION COMPUTER PROGRAMS
(Continue..)

• Technique 8

Use a commercial simulation package to reduce the

amount of programming required.
On the other hand, care must be taken when using a
simulation package because
it contain powerful high-level macro statements,
which are sometimes not well
documented.
TECHNIQUES FOR INCREASING MODEL VALIDITY AND
CREDIBILITY
we discuss six classes of techniques for increasing the validity and
the credibility of a simulation model :

• Collect High-Quality Information and Data on the System

 The analyst will have to be resourceful to obtain a complete and
accurate set
of information.

 Care must be taken to identify the true SMEs for each

subsystem
and to avoid obtaining biased data
TECHNIQUES FOR INCREASING MODEL VALIDITY AND
CREDIBILITY
• Collect High-Quality Information and Data on the System
 Observations of the System :
The modelers need to make sure that the data requirements
(type, format, amount,… etc.) are specified precisely to the people
who provides the data.

 The modelers need to understand the process that produced the

data, rather than treat the observations as just abstract numbers.
 TECHNIQUES FOR INCREASING MODEL VALIDITY AND
CREDIBILITY

• Collect High-Quality Information and Data on the System

 Relevant Results from Similar Simulation Studies

 Experience and Intuition of the Modelers

It will often be necessary to use one’s experience
or intuition to
hypothesize how certain components of a complex
system operate
TECHNIQUES FOR INCREASING MODEL VALIDITY AND
CREDIBILITY

• Maintain a Written Assumptions Document and Perform a

Structured Walk-Through

An assumptions document is not an “exact” description of how the system

works,
but rather a description of how it works relative to the particular issues
that the
model is to address.

The assumptions document should be written to be readable by analysts,

SMEs,
and technically trained managers alike, and it should contain the following:
TECHNIQUES FOR INCREASING MODEL VALIDITY AND
CREDIBILITY
• Maintain a Written Assumptions Document and Perform a
Structured Walk-Through
 An overview section that discusses overall project goals, the specific issues to be
addressed by the simulation study, model inputs, and the performance measures for
evaluation.

 A process-flow or system-layout diagram.

 Detailed descriptions of each subsystem in bullet format and how these subsystems
interact.

 What simplifying assumptions were made and why.

 Limitations of the simulation model.
 Summaries of a data set such as its sample mean and a histogram
TECHNIQUES FOR INCREASING MODEL VALIDITY AND
CREDIBILITY

• Validate Components of the Model by Using Quantitative

Techniques

The simulation analyst should use quantitative

techniques whenever possible to test the validity of
various components of the overall model.

An important technique for determining which model

factors have a significant impact on the desired
measures of performance is sensitivity analysis.
TECHNIQUES FOR INCREASING MODEL VALIDITY AND
CREDIBILITY

• Validate the Output from the Overall Simulation Model

We will discuss several ways for results validation:

 Comparison with an Existing System

If a system similar to the proposed one now exists, then a simulation
model of the existing system is developed and its output data are
compared to those from the existing system itself.

 Comparison with Expert Opinion

If the simulation results are consistent with perceived system behavior,
then
the model is said to have face validity.
TECHNIQUES FOR INCREASING MODEL VALIDITY AND
CREDIBILITY

• Validate the Output from the Overall Simulation Model

 Comparison with Another Model
Suppose that another model was developed for the same system and for a “similar”
purpose,
and that it is thought to be a “valid” representation.

Then numerical statistics or graphical plots for the model that is currently of interest can be
informally compared with the comparable statistics or graphical plots from the other
model.

The confidence-interval procedures can be used to make a more formal comparison

between
the two models.
Simulation and
Modeling
Lecture #5
Random Number Generators
By
Ihab Ali, Ph.D., SMIEEE
Professor of Networks and Security
Helwan University
 Random Number Generators
• Most of Communication Systems and Electronic Devices
incorporate randomness in terms of signals, noise and
interference.
• The most known Distribution being the uniform
distribution.
• In this chapter, we will learn how to generate the
uniform distribution so later we will use random variable
theorems to transform from uniform distribution to
other distributions
 LINEAR CONGRUENTIAL GENERATORS

• A set of integers is defined by the recursive formula:

• the modulus
• , the multiplier
• , the increment
• the seed
• Therefore, 0 ≤ Zi ≤ m - 1, and to obtain the desired random numbers Ui (for i = 1, 2, . .
. ) on [0, 1], we let Ui = Zi/m.
• It’s worth mentioning that the above is pseudo random as long as m is finite (and it
surely is). One way to approach true randomness is to take m to be at the largest
possible value.
 Maximizing the span of U[0,1]

• THEOREM 7.1. The LCG defined above has full period if and only if the following three
conditions hold:
• (a) The only positive integer that (exactly) divides both m and c is 1. (relatively prime)
• (b) If q is a prime number that divides m, then q divides a - 1.
• (c) If 4 divides m, then 4 divides a - 1.
• For a 64-bit processor, m could be as large as .
 prime modulus multiplicative LCGs (PMMLCGs)
• Multiplicative LCGs are advantageous in that the addition of c is not
needed; i.e., c=0.
• Instead of letting m = , it was proposed that m be the largest prime
number that is less than .
• For example, in the case of b = 31, the largest prime that is less than
is, very agreeably, - 1 = 2,147,483,647.
• Please do a search for the largest prime number * that is less than .
• Now for m prime, it can be shown that the period is m - 1 if a is a
primitive element modulo m; that is, the smallest integer l for which - 1
is divisible by m is l = m – 1.
• Two particular values of a that have been widely used for the modulus
m* = - 1 are = = 16,807 and = 630,360,016, both of which are
primitive elements modulo m*.
• Please do a search for appropriate values of the multiplier that are
primitive elements modulo **.