IE406, I & IE

UTK

Simulation
Random-Number
Generation

Dr. Xueping Li
University of Tennessee

Based on
Banks, Carson, Nelson & Nicol
Discrete-Event System Simulation
Purpose & Overview

„ Discuss the generation of random numbers.

„ Introduce the subsequent testing for

randomness:
… Frequency test
… Autocorrelation test.

2
 Random Numbers

„ Why is random number generation necessary?

… In a simulated system, there are inherently random
components that require a method of obtaining numbers that
are random.
… For example - a queuing system model requires interarrival
times, service times, etc., that were drawn from some
specified distribution.

This section focuses on the generation of random numbers,

while the following section will focus on the generation of
random variates...

3
Properties of Random Numbers
„ Two important statistical properties:
… Uniformity
… Independence.

„ Random Number, Ri, must be independently drawn from a

uniform distribution with pdf:

⎧1, 0 ≤ x ≤ 1
f ( x) = ⎨
⎩0, otherwise
1
2
1 x
∫
1
E ( R) = xdx = =
0 2 2
0
Figure: pdf for
random numbers
4
 Properties of
Random Numbers

„ Independence - the probability of observing a value

in a particular interval is independent of the previous
value drawn.
„ Expected value and variance of random numbers
… Expected value of each Ri is given by
21
1 x 1
E ( R ) = ∫ xdx = =
0 2 0
2
… Variance for each Ri is given by

31 2
1 x ⎛ 1⎞ 1 1 1
V ( R ) = ∫ x dx − [ E ( R )] =
2 2
−⎜ ⎟ = − =
0 3 0 ⎝2⎠ 3 4 12

5
Generation of Pseudo-Random Numbers
„ “Pseudo”, because generating numbers using a known
method removes the potential for true randomness.
„ Goal: To produce a sequence of numbers in [0,1] that
simulates, or imitates, the ideal properties of random numbers
(RN).
„ Important considerations in RN routines:
… Fast
… Portable to different computers
… Have sufficiently long cycle
… Replicable
… Closely approximate the ideal statistical properties of uniformity
and independence.

6
Techniques for Generating Random
Numbers

„ Midsquare Generator
„ Linear Congruential Method (LCM).
„ Combined Linear Congruential Generators (CLCG).
„ Random-Number Streams.

7
 Midsquare Generator

Random number seed - initial random number used to

generate the next random number which is in turn
transformed into the new seed value.
„ Proposed by von Neumann and Metropolis in the
1940’s
… Start with an initial four-digit positive integer seed X0
… Square it to obtain an integer with up to eight digits
… Take the middle four digits as the next four-digit number, X1
… Place a decimal point at the left of X1 to get first random
number R1
… Square X1 and continue process

8
 Midsquare Generator -
Example

Let X0 = 5497
„ X1 = (5497)2 = 30217009 X1 = 2170, R1 = 0.2170
„ X2 = (2170)2 = 04708900 X2 = 7089, R2 = 0.7089
„ X3 = (7089)2 = 50253921 X3 = 2539, R3 = 0.2539
„ ...and so on...

Pros of the midsquare method

… Rather simple to implement
Cons of the midsquare method
… Difficult to choose initial seed that will give good sequence
… Strong tendency to degenerate fairly rapidly to zero

9
 Midsquare Generator -
Poor Examples

Let X0 = 5197
„ X1 = (5197)2 = 27008809 X1 = 0088, R1 = 0.0088
„ X2 = (0088)2 = 00007744 X2 = 0077, R2 = 0.0077
„ X3 = (0077)2 = 00005929 X3 = 0059, R3 = 0.0059
„ X4 = (0059)2 = 00003481 X4 = 0034, R4 = 0.0034
0 Let X0 = 6500
„ X1 = (6500)2 = 42250000 X1 = 2500, R1 = 0.2500
„ X2 = (2500)2 = 06250000 X2 = 2500, R2 = 0.2500

10
More Examples
Seed = 9713
1 94342369 2369
2 11716929 6929
Seed = 2784 51394561 4561
3
1 7750656 7506 4 15563025 3025
2 56340036 3400 5 31696900 6900
6 48566961 6961
3 11560000 5600 *
7 32137561 7561
4 31360000 3600 8 1890625 625
5 12960000 9600 9 79316836 6836
6 92160000 1600 10 10036224 6224
11 131044 1044
7 2560000 5600 * 1716100 6100
12
8 31360000 3600
9 12960000 9600 84 7290000 2900
85 8410000 4100 *
10 92160000 1600 86 16810000 8100
11 2560000 5600 * 87 65610000 6100
88 37210000 2100
89 4410000 4100 *

11
Linear Congruential Method [Techniques]

„ To produce a sequence of integers, X1, X2, … between 0

and m-1 by following a recursive relationship:

X i +1 = (aX i + c) mod m, i = 0,1,2,...

The The The

multiplier increment modulus

„ The selection of the values for a, c, m, and X0 drastically

affects the statistical properties and the cycle length.
„ The random integers are being generated [0,m-1], and to
convert the integers to random numbers:
Xi
Ri = , i = 1,2,...
m

12
Example [LCM]

„ Use X0 = 27, a = 17, c = 43, and m = 100.

„ The Xi and Ri values are:
X1 = (17*27+43) mod 100 = 502 mod 100 = 2, R1 = 0.02;
X2 = (17*2+32) mod 100 = 77, R2 = 0.77;
X3 = (17*77+32) mod 100 = 52, R3 = 0.52;
…

13
Another example

a= 17 c = 9
seed= 27 m = 10000

1 468 468 0.0468

2 7965 7965 0.7965 180
3 135414 5414 0.5414 160
4 92047 2047 0.2047 140
5 34808 4808 0.4808 120

Frequency
6 81745 1745 0.1745
100
7 29674 9674 0.9674
8 164467 4467 0.4467 80
9 75948 5948 0.5948 60
10 101125 1125 0.1125 40
11 19134 9134 0.9134 20
12 155287 5287 0.5287
0
13 89888 9888 0.9888
0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 More
14 168105 8105 0.8105
15 137794 7794 0.7794 Random Number
16 132507 2507 0.2507
17 42628 2628 0.2628
18 44685 4685 0.4685
19 79654 9654 0.9654
20 164127 4127 0.4127

Period = 2000
14
 Other Types of Generators

„ Quadratic Congruential Generator

… Xnew = (a1 * Xold2 + a2 * Xold + c) mod m
„ Combination of generators
… Shuffling – shuffle the output from the first LCG with a
second LCG
… L’Ecuyer – takes integers generated from two different LCGs
with different moduli, then Z i = ( Z1i − Z 2i ) mod m
… Wichman/Hill – takes the fractional parts of multiple random
numbers generated from separate generators
„ Trausworthe generator
… Generates sequence of random bits

15
Characteristics of a Good Generator
[LCM]
„ Maximum Density
… Such that he values assumed by Ri, i = 1,2,…, leave no large
gaps on [0,1]
… Problem: Instead of continuous, each Ri is discrete
… Solution: a very large integer for modulus m
„ Approximation appears to be of little consequence
„ Maximum Period
… To achieve maximum density and avoid cycling.
… Achieve by: proper choice of a, c, m, and X0.
„ Most digital computers use a binary representation of
numbers
… Speed and efficiency are aided by a modulus, m, to be (or close
to) a power of 2.

16
Combined Linear Congruential Generators
[Techniques]

„ Reason: Longer period generator is needed because of the

increasing complexity of stimulated systems.
„ Approach: Combine two or more multiplicative congruential
generators.
„ Let Xi,1, Xi,2, …, Xi,k, be the ith output from k different
multiplicative congruential generators.
… The jth generator:
„ Has prime modulus mj and multiplier aj and period is mj-1

„ Produces integers Xi,j is approx ~ Uniform on integers in [1,

m-1]
„ Wi,j = Xi,j -1 is approx ~ Uniform on integers in [1, m-2]

17
Combined Linear Congruential Generators
[Techniques]
… Suggested form:
⎧ Xi
⎪⎪ m , Xi ; 0
⎛ k ⎞
X i = ⎜ ∑ (−1) X i , j ⎟⎟ mod m1 − 1
⎜ j −1
Hence, Ri = ⎨ 1
⎝ j =1 ⎠ m −1
⎪ 1 , Xi = 0
⎪⎩ m1
The coefficient:
Performs the
subtraction Xi,1-1

„ The maximum possible period is:

(m1 − 1)(m2 − 1)...(mk − 1)

P=
2 k −1

18
Combined Linear Congruential Generators
[Techniques]
„ Example: For 32-bit computers, L’Ecuyer [1988] suggests combining
k = 2 generators with m1 = 2,147,483,563, a1 = 40,014, m2 =
2,147,483,399 and a2 = 20,692. The algorithm becomes:
Step 1: Select seeds
„ X1,0 in the range [1, 2,147,483,562] for the 1st generator

„ X2,0 in the range [1, 2,147,483,398] for the 2nd generator.

Step 2: For each individual generator,

X1,j+1 = 40,014 X1,j mod 2,147,483,563
X2,j+1 = 40,692 X1,j mod 2,147,483,399.
Step 3: Xj+1 = (X1,j+1 - X2,j+1 ) mod 2,147,483,562.
Step 4: Return ⎧ X j +1
⎪⎪ , X j +1 > 0
R j +1 = ⎨ 2,147,483,563
⎪ 2,147,483,562 , X j +1 = 0
⎪⎩ 2,147,483,563
Step 5: Set j = j+1, go back to step 2.
… Combined generator has period: (m1 – 1)(m2 – 1)/2 ~ 2 x 1018
19
Random-Numbers Streams [Techniques]

„ The seed for a linear congruential random-number generator:

… Is the integer value X0 that initializes the random-number sequence.
… Any value in the sequence can be used to “seed” the generator.
„ A random-number stream:
… Refers to a starting seed taken from the sequence X0, X1, …, XP.
… If the streams are b values apart, then stream i could defined by starting
seed: Si = X b (i −1)

… Older generators: b = 105; Newer generators: b = 1037.

„ A single random-number generator with k streams can act like k
distinct virtual random-number generators
„ To compare two or more alternative systems.
… Advantageous to dedicate portions of the pseudo-random number
sequence to the same purpose in each of the simulated systems.

20
Tests for Random Numbers
„ Two categories:
… Testing for uniformity:
H0: Ri ~ U[0,1]
H1: Ri ~/ U[0,1]
„ Failure to reject the null hypothesis, H0, means that evidence of
non-uniformity has not been detected.
… Testing for independence:
H0: Ri ~ independently
H1: Ri ~/ independently
„ Failure to reject the null hypothesis, H0, means that evidence of
dependence has not been detected.
„ Level of significance α, the probability of rejecting H0 when it
is true: α = P(reject H0|H0 is true)

21
Tests for Random Numbers
„ When to use these tests:
… If a well-known simulation languages or random-number
generators is used, it is probably unnecessary to test
… If the generator is not explicitly known or documented, e.g.,
spreadsheet programs, symbolic/numerical calculators, tests
should be applied to many sample numbers.
„ Types of tests:
… Theoretical tests: evaluate the choices of m, a, and c without
actually generating any numbers
… Empirical tests: applied to actual sequences of numbers
produced. Our emphasis.

22
Frequency Tests [Tests for RN]

„ Test of uniformity
„ Two different methods:
… Kolmogorov-Smirnov test
… Chi-square test

23
Kolmogorov-Smirnov Test [Frequency Test]

„ Compares the continuous cdf, F(x), of the uniform

distribution with the empirical cdf, SN(x), of the N sample
observations.
… We know: F ( x ) = x, 0 ≤ x ≤ 1
… If the sample from the RN generator is R1, R2, …, RN, then the
empirical cdf, SN(x) is:
number of R1 , R2 ,..., Rn which are ≤ x
S N ( x) =
N

„ Based on the statistic: D = max| F(x) - SN(x)|

… Sampling distribution of D is known (a function of N, tabulated in
Table A.8.)
„ A more powerful test, recommended.

24
Kolmogorov-Smirnov Test [Frequency Test]

„ Example: Suppose 5 generated numbers are 0.44, 0.81, 0.14,

0.05, 0.93.
Arrange R(i) from
Step 1: R(i) 0.05 0.14 0.44 0.81 0.93 smallest to largest

i/N 0.20 0.40 0.60 0.80 1.00

D+ = max {i/N – R(i)}
i/N – R(i) 0.15 0.26 0.16 - 0.07
Step 2:
R(i) – (i-1)/N 0.05 - 0.04 0.21 0.13
D- = max {R(i) - (i-1)/N}

Step 3: D = max(D+, D-) = 0.26

Step 4: For α = 0.05,
Dα = 0.565 > D

Hence, H0 is not rejected.

25
Chi-square test [Frequency Test]

„ Chi-square test uses the sample statistic:

n is the # of classes Ei is the expected
n
(Oi − Ei ) 2 # in the ith class
Χ =∑
2
0
i =1 Ei Oi is the observed
# in the ith class

… Approximately the chi-square distribution with n-1 degrees of

freedom (where the critical values are tabulated in Table A.6)
… For the uniform distribution, Ei, the expected number in the each
class is: N
Ei = , where N is the total # of observation
n

„ Valid only for large samples, e.g. N >= 50

26
Tests for Autocorrelation [Tests for RN]

„ Testing the autocorrelation between every m numbers

(m is a.k.a. the lag), starting with the ith number
… The autocorrelation ρim between numbers: Ri, Ri+m, Ri+2m,
Ri+(M+1)m
… M is the largest integer such that i + (M + 1 )m ≤ N

„ Hypothesis:
H 0 : ρ im = 0, if numbers are independen t
H 1 : ρ im ≠ 0, if numbers are dependent

„ If the values are uncorrelated:

… For large values of M, the distribution of the estimator of ρim,
denoted ρ̂ im is approximately normal.

27
Tests for Autocorrelation [Tests for RN]

„ Test statistics is: ρˆ im

Z0 =
σˆ ρˆ im

… Z0 is distributed normally with mean = 0 and variance = 1, and:

1 ⎡M ⎤
ˆρim = ⎢ ∑
M + 1 ⎣ k =0
Ri + km Ri +(k +1 )m ⎥ − 0.25
⎦
13M + 7
σˆ ρim =
12(M + 1 )
„ If ρim > 0, the subsequence has positive autocorrelation
… High random numbers tend to be followed by high ones, and vice versa.
„ If ρim < 0, the subsequence has negative autocorrelation
… Low random numbers tend to be followed by high ones, and vice versa.

28
Example [Test for Autocorrelation]

„ Test whether the 3rd, 8th, 13th, and so on, for the
following output on P. 265.
… Hence, α = 0.05, i = 3, m = 5, N = 30, and M = 4

1 ⎡(0.23)(0.28) + (0.25)(0.33) + (0.33)(0.27 ) ⎤

ρˆ 35 = ⎥ − 0.25
4 + 1 ⎢⎣ + (0.28)(0.05) + (0.05)(0.36) ⎦
= −0.1945
13( 4) + 7
σˆ ρ35 = = 0.128
12( 4 + 1 )
0.1945
Z0 = − = −1.516
0.1280

… From Table A.3, z0.025 = 1.96. Hence, the hypothesis is not

rejected.
29
Shortcomings [Test for Autocorrelation]

„ The test is not very sensitive for small values of M,

particularly when the numbers being tests are on the low
side.
„ Problem when “fishing” for autocorrelation by performing
numerous tests:
… If α = 0.05, there is a probability of 0.05 of rejecting a true
hypothesis.
… If 10 independence sequences are examined,
„ The probability of finding no significant autocorrelation, by
chance alone, is 0.9510 = 0.60.
„ Hence, the probability of detecting significant autocorrelation
when it does not exist = 40%

30
Summary
„ In this chapter, we described:
… Generation of random numbers
… Testing for uniformity and independence

„ Caution:
… Even with generators that have been used for years, some of
which still in used, are found to be inadequate.
… This chapter provides only the basic
… Also, even if generated numbers pass all the tests, some
underlying pattern might have gone undetected.

31