Simulation

Chapter 5

Randomness in Simulation
Plan
 Objective
 Random Numbers
 Random Number Generators (LCG, CMRG)
 Variate Generation
 Variance Reduction
Objective

 Understand the Randomness behind simulation

 Be able to manipulate a simple random number generator

 Use Common Random Number Method for robust comparison

of simulation output
Random Numbers

A random number is a number generated by a process

whose outcome is unpredictable.

 Random numbers are the basic building blocks for all

simulation algorithms.

 Basis for generating observations from all other

distributions and random processes
 Random Numbers
 Goal: To produce Pseudo-Random numbers between [0, 1] that simulates,
or replicates, the ideal properties of random numbers.

 The word “Pseudo” is used, because generating numbers using a known

method (algorithm) removes the potential for true randomness.

 The important considerations in selecting Random Number generators are:

 Fast
 Have sufficient long cycle (period)
 Replicable
 Closely approximate the ideal statistical properties of uniformity and
independence
Random Number Generators : RNGs

 The most widely used RNGs are:

 Linear Congruential Generator (LCG): Lehmer [1951]

 Combined Multiple Recursive Generator(CMRG): L’Ecuyer [1996], SIMAN-

ARENA, MATLAB, R, SPSS, etc.
 The LCG
 This
method produces a sequence of integers, X1, X2 … between 0
and m-1 by following a recursive relationship:

Χ𝑖 = 𝑎 Χ𝑖−1 + 𝑐 𝑚𝑜𝑑 𝑚 𝑖 = 0,1,2, …

The multiplier The increment The modulus
 a, c, and m are constants

 Specify the initial Value the seed X0

 The Random variable Ui is generated by Ui =Xi/m

Use the LCG with : a= 3, c=3, m=5 and the seed X0=3 to generate
random variable Ui.
 The LCG
 The selection of the values for a, c, m drastically affects the cycle length.

 Test the 1st LCG: m = 32, a = 5, c = 6, seed X0 = 1:

Xi = (5 Xi–1 + 6) mod 32

 Test the 2nd LCG: : m = 32, a = 5, c = 5, seed X0 = 1:

Xi = (5 Xi–1 + 5) mod 32

 Test the 3rd LCG: m = 63, a = 22, c = 4, seed X0 = 19:

Xi = (22 Xi–1 + 4) mod 63

 Test the 4rd LCG: m = 63, a = 2, c = 4, seed X0 = 19:

Xi = (2 Xi–1 + 4) mod 63

See LCG.xls
Issues with LCGs
 Theorem:
In order to have a sequence of maximal length i.e. length
equal to m, 3 conditions must be verified:
i) “c” and “m” are relatively prime, i.e., the only positive
integer that exactly divides both m and c is 1.
ii) (a-1) is a multiple of each prime number which divides
“m” (including m if m is prime).
iii) if “m” is a multiple of 4 then (a-1) is also a multiple of 4
Issues with LCG
 Cycle length: < m
 Typically, m = 2.1 billion (m = 231 )or more
 Other parameters chosen so that cycle length = m

 Statistical properties
 Uniformity, independence
 There are many tests
 Empirical tests
 K-S tests
Issues with LCG
 Suppose 5 generated numbers are 0.44, 0.81, 0.14, 0.05, and 0.93. It is desired to
perform a test for uniformity using Kolmogorov-Smirnov test with a level of
significance α = 0.05.
Issues with LCGs
 “Regularity”
of LCGs (and other kinds of RNGs); for
example earlier LCG :

Plot of Ui vs. i Plot of Ui+1 vs. Ui

 Solution:
Other kinds of RNGs — longer memory in
recursion, combination of several LCGs
Combined Multiple Recursive Generator(CMRG)
 Uses some of same ideas as LCG
 Modulo division, recursive on earlier values
 But is not an LCG
 Combines two separate generators

 Combined multiple recursive generator (CMRG)

An = (1403580 An-2 – 810728 An-3) mod 4294967087 Two

simultaneous
recursions
Bn = (527612 Bn-1 – 1370589 Bn-3) mod 4294944443

Zn = (An – Bn) mod 4294967087 Combine the two

Zn / 4294967088 if Zn > 0 Next

Un = random
4294967087 / 4294967088 if Zn = 0 number

Seed = a six-vector of first three An’s, Bn’s

Variate Generation
 We seek to generate random variables X distributed according to any
other theoretical distribution (Exponential, Triangular, etc.)
 The most general and widely used method is the Inverse Transform
method, Banks et al. (1999).
 Based on the following Algorithm:
 Variate Generation
 Example : generate Xi according to the exponential distribution:
Generate uniform random numbers U1, U2, U3 … and compute desired
random variate by Xi = F-1(Ui)
We have to solve the equation : F(X) = U
For exponential distribution, the solution is as follows
1- e-λx = U
e-λx = 1 - U
-λX = ln (1 - U)
X=-1/λ ln(1-U)
Thus : Xi =-1/λ ln(1-Ui) for i= 1,2,3,…
Xi =-1/λ ln(Ui)
(Replace 1-Ui by Ui as both are uniformly distributed on (0,1))
Variance Reduction

 Theennemy in stochastic simulation: the more variance the less

precise are your output.

 Usingthe technique of Common Random Numbers ( CRN )can

drastically decrease output variance when the objective is to
compare two alternatives.

 See Model 12-02.doe

“Natural” Comparison
 Run case A, make change to get to case B (3.5% more capacity)
and run it, then Compare (A-B) Means via Output Analyzer:

 Difference is not statistically significant (A-B encloses 0)

 Solution Variance Reduction by using the same Random numbers stream
in each source of randomness
Synchronization of Random Numbers in CRN
 Synchronize by source of randomness
 Assign stream to each point of variate generation
 Model12-1: 14 sources of randomness, separate stream for each modify into
Model 12-2
 Effect of CRN
“Natural”
Comparison

Synchronized
CRN

 The difference is NOW SIGNIFICANT !