Random Numbers for Simulations

Pseudo Random Number Generators

Roberto Innocente

SISSA, Trieste
for the joint ICTP/SISSA MHPC.it master course

Apr 4, 2017 - Revised version

Roberto Innocente Random Numbers for Simulations

Where it all began ?

At the end of 1700 George Leclerc, Comte de Buffon

tried to compute π with random experiments.

Roberto Innocente Random Numbers for Simulations

George-Louis Leclerc, Comte de Buffon, 1777 :
Montecarlo method ante-litteram

A needle of length 1 is thrown over a lined paper with lines every 1

unit . The probability that it hits a line can be computed as :

cos(θ) π
Z π  
sin(θ)
Shaded Portion Area = dθ = − =1
θ=0 2 2 0

Shaded Portion Area 1 2

Prob = = → π=
Area of rectangle π/2 Prob

Roberto Innocente Random Numbers for Simulations

Example : Buffon’s needle in R

# R code
buffonp <- function ( n ) {
k =0;
for ( i in 1: n ) {
theta = runif (1 , min =0 , max = pi )
y = runif (1 , min =0 , max =1/2) ;
if ( y +1/2* sin ( theta ) >1/2) k = k + 1 }
return ( k / n )
}
for ( i in 1:6) {
w =10^ i ; bp = buffonp ( w )
cat ( ’ rn = ’ ,w , ’ computed pi = ’ ,2.0/ bp , ’ error = ’ , abs ( pi -2.0/ bp
) , ’\n ’)
}
# rn = 10 computed pi = 2 error = 1.141593
# rn = 100 computed pi = 3.389831 error = 0.2482379
# rn = 1000 computed pi = 3.04878 error = 0.09281217
# rn = 10000 computed pi = 3.134305 error = 0.007287686
# rn = 1 e +05 computed pi = 3.138584 error = 0.003008469
# rn = 1 e +06 computed pi = 3.144595 error = 0.003002102

Roberto Innocente Random Numbers for Simulations

RNG, TRNG, PRNG I

TRNG (True Random Number Generators)

noise based e.g from atmospheric noise
https://www.random.org/
free running oscillator (FRO) : simplest and cheapest way
chaos based
quantum based e.g. Geiger counters, fluctuations of vacuum
https://qrng.anu.edu.au/
PRNG (Pseudo Random Number Generators) or simply RNG
algorithmic
We want to be able to produce RN fast, in a cheap way, we need
repeatability : we need algorithmic RNG !!.
There is a subclass of PRNG that will not be covered here and it is
the class of cryptographically robust PRNG. These require the
special quality of unpredictability that is computationally expensive
and is never shown by the efficient PRNG we need for simulations.

Roberto Innocente Random Numbers for Simulations

Divide and conquer

Generation of Random Numbers is usually splitted into :

1 Generation of uniformly distributed integers xi in
[0 . . . (m − 1)]
2 Mapping of integers in [0 . . . (m − 1)] to uniformly distributed
xi
reals in U(0, 1) using ui = m . In many cases the 1st step is
allowed to produce 0 while usually we want the 2nd step not
xi +1
to produce it. Therefore often is used ui = m+1 .
3 Mapping of uniformly distributed reals in U (0, 1) to the
wanted CDF (Cumulative Distribution Function) F (x) using
most of the time its inverse F −1 (x)
This is the reason why we will center our discussion about uniform
random numbers on (0, 1): ∼ U (0, 1).

Roberto Innocente Random Numbers for Simulations

How to map a uniform variate or deviate ∼ U (0, 1) in a
differently distributed variate? I

We say that a sequence of numbers is a sample from a cumulative

distribution function F , if they are a realization of a rv with CDF
F . If F is the uniform distribution over (0, 1) then we call the
samples from F uniform deviates/variates and we write ∼ U (0, 1).
Theorem : inversion
Suppose U ∼ U (0, 1) and F to be a continuos strictly increasing
cumulative distribution function (CDF). Then F −1 (U) is a sample
from F .
Transform to a Normal deviate :
Box-Muller transform is more efficient:
1 generate U1 ∼ U (0, 1) and U2 ∼ U (0, 1)
√
2 θ = 2πU2 , ρ = −2 log U1
3 Z1 = ρ cos θ is a normal variate

Roberto Innocente Random Numbers for Simulations

Example in R

# exponential distribution : R code

Exponential distr: l*exp(−l*x)
pdf ( file = ’ exp . pdf ’)
lambda = 1; x = seq (0 ,5 ,0.05)
y = exp ( - lambda * x ) ; z =1 - exp ( - lambda * x )
1.0

plot (x ,y , type = ’n ’)
title ( ’ Exponential distr : l * exp ( - l * x
0.8

Exponential CDF ) ’)
lines (x ,y , col = ’ red ’) ; lines (x ,z , col
= ’ green ’)
0.6

text (3 ,0.5 , ’ RN obtained from uniform

RN obtained from uniform deviate using inversion deviate using inversion ’)
y

text (2 ,0.8 , ’ Exponential CDF ’ , col = ’

0.4

green ’)
text (2 ,0.1 , ’ Exponential pdf ’ , col = ’
red ’)
0.2

Exponential pdf
invcdf < - function ( yy ) { lambda = 1;
xx = - log (1 - yy ) / lambda ;
0.0

0 1 2 3 4 5
return ( xx ) ; }
w = runif (1000) ; ic = invcdf ( w )
x
lines ( ecdf ( ic ) , xlim = c (0 ,5) , ylim = c
(0 ,1) )

Roberto Innocente Random Numbers for Simulations

Qualities of Good RNG

Good Theoretical Basis

Long Period
”pass” Empirical Tests
Efficient
Repeatable
Portable

Roberto Innocente Random Numbers for Simulations

Theoretical framework

Theoretical Framework for Random Number Generators :

S
initial state
Set of states

s0=initial state

current state next state

g:S-->(0,1) f:S-->S
output transition
function function

(0,1)

S set of all states/seeds (e.g. 1 integer -> 2ˆ32 states)

f:S-->S transition function that moves the rng to the next state
g:S-->(0,1) output function that from a state outputs a number in the (0,1) interval

An upper bound on the period of the generator is the cardinality of S : |S|

Roberto Innocente Random Numbers for Simulations

Pre-period, Period

There can be confusion about these terms, they refer to the general
case in which a generator can cycle skipping some initial outcomes.

Roberto Innocente Random Numbers for Simulations

History of field based on scholars leaders

Von Neumann was maybe the first to devise an algorithm : the

middlesquare method. A few leaders in the field during more than
75 years of electronic computing were in succession :
Donald Knuth
George Marsaglia
Pierre L'Ecuyer

Roberto Innocente Random Numbers for Simulations

Donald Knuth

born 1938, PhD at Caltech, worked at Stanford,

now Professor Emeritus at Stanford
Writer of the first bible of algorithms. 3 and now
4 books nick-named TAOCP: The art of
computer programming.
Creator of TEX and METAFONT and the
Computer Modern family of fonts. Writer of the
5 books about them : Computer and
typesetting: A,B,C,D,E
Volume 2 of The Art Of Computer
Programming : Seminumerical Algorithms(1998)
dedicates the 189 pages of chapter 3 to Random
Numbers. In it there is also the description of a
battery of rng tests.

Roberto Innocente Random Numbers for Simulations

Knuth reports his attempt at a Super-random ng.
Sometimes complexity hides simple behaviour

The first time Knuth ran this program, it converged quickly to

6065038420 (a fixed point of the algorithm). After this time it was
mainly converging to a cycle of length 3178 !
Roberto Innocente Random Numbers for Simulations
Linear Congruential Generators or LCG I
LCG notation
xn+1 ≡ (a ∗ xn + c) (mod m)
will be indicated by LCG (m, a, c, x0 ). m is called the modulus, a
the multiplier , c the increment, x0 the starting value or seed. We
use c like Knuth does and we set b = a − 1 for convenience.
Introduced by Lehmer in 1949. Sometimes when c = 0 they are
called Multiplicative LCG or MLCG and denoted by
MLCG (m, a). When c 6= 0 Mixed Linear Congruential.
Lehmer generator is u0 6= 0, un+1 ≡ (23 ∗ un ) (mod 108 + 1)
ANSIC
LCG (231 , 1103515245, 12345, 12345) Super-duper LCG (232 , 69069, 0, 1)
MINSTD LCG (231−1 , 75 , 0, 1) NAG LCG (259 , 1313 , 0, 232 + 1)
RANDU LCG (231 , 216 , 0, 1) DRAND48
LCG (248 , 25214903917, 11, 0)
APPLE LCG (235 , 513 , 0, 1)
Roberto Innocente Random Numbers for Simulations
Linear Congruential Generators or LCG II
c = 0 takes less time to compute, but cuts down the period of
the sequence that anyway can still be long
it can be proved that

xn+k ≡ (ak ∗ xn + (ak − 1)c/b) (mod m)

that expresses the n + k term in terms of the n term. In

particular respect to x0 .

xk ≡ (ak ∗ x0 + (ak − 1)c/b) (mod m)

That is: the subsequence consisting of every k th term is also

an LC sequence.
Choice of m : should be large because it’s a limit for the
period ρ of the rng, should make it simple to compute
(a ∗ xn + c) (mod m),

Roberto Innocente Random Numbers for Simulations

Linear Congruential Generators or LCG III
MLCG (m, a) :
If m is prime and a is a primitive root of m and x0 6= 0 then the
sequences {xn } are periodic with period length ρ = m − 1 and the
generator is called a full period MLCG.
If m = 2w then the maximal period is ρ = 2w −2 = m/4 and is
attained in particular when a ≡ 5 (mod 8).

[14] We want a large m to make the grid of RN finer. But we need

to keep m not larger than a computer word so that we can do
operations efficiently. Therefore we choose m ≤ 232 for 32-bit
processors or m ≤ 264 for 64-bit processor. The theory is nicer if m
is prime or is a power of 2 like 2w . Common choices:
32-bit proc 64-bit proc
Prime 2k Prime 2k
m = 231 − 1 m = 232 m = 248 − 59 m = 264
m = 263 − 25
m = 264 − 59
Roberto Innocente Random Numbers for Simulations
Linear Congruential Generators or LCG IV

LCG (m, a, c, x0 ) :
An LCG has full period m if and only if :
1 The GCD(Greatest Common Divisor) of m and c is 1.
2 if q is a prime that divides m then q divides (a − 1).
3 if 4 divides m, then 4 divides (a − 1)
(Hull-Dobell Theorem)

A lot of work has been done on these generators especially to give

multipliers that provide as little as possible of Marsaglia’s effect.
You can’t use them without reading [14] that for common word
sizes computes the highest prime smaller than the largest integer
and gives good multipliers , e.g.:

Roberto Innocente Random Numbers for Simulations

 Figure: from l'Ecuyer
1988, MLCG :
m = 2e , c = 0

Roberto Innocente Random Numbers for Simulations

 Figure: from l'Ecuyer
1988, LCG : m = 2e ,c
odd

Roberto Innocente Random Numbers for Simulations

 Figure: l'Ecuyer
1988, LCG : m prime

Roberto Innocente Random Numbers for Simulations

LCG : low order bits are less random

Figure: Lowest order bit 256x256 b0 , 256x256 third bit b2 , 256x256

successive bits

Roberto Innocente Random Numbers for Simulations

George Marsaglia

born 1924, † 2011, PhD Ohio State, then

University of Florida, University of Washington
discovered what is called Marsaglia’s effect. The
successive n-tuples generated by Linear
Congruential Generators (LCG) lie on a small
number of equally spaced hyperplanes in
n-dimensional space.
developed the diehard statistical tests for rng,
1996
developed many of the well known methods for
generating rn : multiply-with-carry, subtract
with borrow, xorshift , KISS93, KISS99, . . .

Roberto Innocente Random Numbers for Simulations

Fibonacci, Lagged Fibonacci(Marsaglia 1983) I

Probably you know the Fibonacci's sequence : an attempt made by

Leonardo Fibonacci (aka il Pisano, born in Pisa ∼ 1175, † ∼ 1245)
to model the growth of a population of rabbits xn = xn−1 + xn−2 .
Why not to generate rn based on a longer previous history ?
Fibonacci
Xn = Xn−1  Xn−2 (mod m) , denoted F (1, 2, )

has poor distribution qualities.

Lagged Fibonacci

Xn = Xn−k  Xn−l (mod m) , denoted F (k, l, )

 is a generic operator from +, −, ∗, ⊕. ⊕ is the XOR binary

operator. k , l are called lags . Lags larger then 16 produce good
rng. k = 24, l = 55 was studied extensively, like 30, 127. They

Roberto Innocente Random Numbers for Simulations

Fibonacci, Lagged Fibonacci(Marsaglia 1983) II

were used extensively, but in the ' 90 it was discovered that they
fail a famous test of randomness (but a workaround exists). The
one proposed by Marsaglia is xn ≡ xn−5 + xn−17 (mod 2k ). Period
of this lagged Fibonacci is 2k ∗ (217 − 1), quite longer than LCGs..
State is an array of 17 integers.

Roberto Innocente Random Numbers for Simulations

Marsaglia’s theorem

Roberto Innocente Random Numbers for Simulations

Lattice structure of LCG generators
Marsaglia's effect. Successive t-uples obtained from an LCG
generator fall on, at most, (t!m)1/t parallel hyperplanes, where m
is the modulus used in the LCG(marsaglia1968) :
2D Lattice structure of (31*x) %% (2^16) 3D Lattice structure of (31*x) %% (2^16)

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
60000

● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●● ●●
● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ●
● ● ● ● ● ● ● ● ●● ● ● ●●
● ●● ● ●
● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ●
●●
●● ●● ● ● ● ● ●●
● ● ● ● ● ● ● ●● ●● ●● ● ●● ● ●● ●●
● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ●●
●●
●●
●● ●● ●●
●● ●● ● ● ● ●● ●●
● ● ● ● ● ● ● ●● ●● ●● ●● ●● ● ● ● ●● ●
● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●●
50000

● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● ●● ● ● ● ●
● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ● ●
● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●●● ●● ●● ●● ●●● ●●● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ●● ● ● ● ●● ●● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ● ●● ● ●●● ● ●● ●
●● ●●● ●● ● ●●● ●
●● ●
● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●
●● ● ●●

10000 20000 30000 40000 50000 60000 70000

● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ● ● ● ●● ● ●● ●● ●● ●● ●● ●● ● ●●
● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ● ●● ●
●● ● ● ● ● ●
●● ●
● ● ●
● ● ● ● ● ● ● ●● ●●● ●● ● ● ● ●● ● ●● ●● ●●● ●● ●
●● ●
● ● ● ● ● ● ● ● ● ● ● ● ●● ●●
●● ●● ●● ●● ●● ●● ●●
●● ●● ●● ●● ●● ●●
●● ●● ● ●●
y[seq(2, n − (n%%2), 2)]

● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● ● ●● ●● ● ●● ●● ●● ●● ●● ●● ●● ●●
● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● ●
● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● ●●●● ● ●● ●● ●● ●● ●● ●● ●●
●● ●● ● ●●
40000

● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● ●● ●● ● ●● ●● ●● ●● ●● ●● ●● ●●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ● ● ● ●● ● ●● ● ● ● ● ●● ● ● ● ● ●
●● ●● ●● ●● ●● ●● ●●● ●●● ●● ● ● ●● ●
●● ●● ●● ●●● ●●● ●●
● ● ● ● ● ● ● ● ● ● ● ● ●●●● ●● ●● ●● ●● ●● ●● ●● ● ● ●
● ●● ●● ●● ●● ●● ●● ●●
● ● ● ● ● ● ● ●● ●● ●● ●● ● ●● ●● ●● ●
● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ●●
●●
●●
●● ●● ●● ●●
●●
●●
●● ●● ●● ●●
●●
● ●
●●
● ●● ●● ●●
●●
●●
●●
●●
●● ●● ● ●●
● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●●
● ●● ●● ●● ●● ●● ●● ●
● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●
● ●● ●● ●● ●● ●● ●● ● ●●
● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ● ●●● ●● ●● ●● ●● ●● ●●
● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●
● ● ● ● ● ● ● ● ● ● ● ● ●● ●●●● ●● ●● ●● ●● ●● ●● ●● ● ●
● ●● ●● ●● ●●
●● ●● ● ●●

y[seq(3, n − (n%%3), 3)]

● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●
● ●● ●● ●● ●● ●● ●●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●
●● ●● ●● ●● ●● ●● ●● ●● ● ●● ●● ●● ●● ●● ● ●●
30000

● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ● ●● ●● ●● ●●

y[seq(2, n − (n%%3), 3)]

● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ● ● ● ●● ●● ● ● ● ●● ● ●
● ● ● ●● ●● ●● ●
● ●● ● ●
●● ●●● ●●● ● ● ● ●●● ●●● ●● ● ●
● ● ●● ● ●● ● ●●● ● ●●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ● ●● ● ●● ●● ●● ●●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●●
●●
●●
●● ● ● ● ● ●
●● ● ●
●● ●●● ● ●●
●● ●●
●● ● ●
●● ● ● ●
● ●
●
●● ● ●
●● ●● ● ● ●●
● ● ● ● ● ● ● ●● ●● ●● ●● ● ●● ●●
● ● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●●
●● ●● ●● ●
●● ●●
●● ●● ●● ●
●● ●●
●●
●●
●● ● ●● ● ●● ● ●●
●● ●● ●● ●
● ●●
● ● ● ● ● ● ● ● ● ● ● ● ●● ●
●● ● ● ● ●● ●●● ●● ● ●● ● ●
● ● ● ● ● ● ● ● ●● ●
●●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●
●● ●● ●● ●● ●
●● ●● ●● ●● ●●● ●● ●
● ●● ● ● ●● ●● ●● ●
● ● ● ● ● ● ● ● ● ● ● ● ●● ● ●● ● ●●
●●
●●
●● ●● ●● ●
●● ●●
●●
●●
●● ●● ●
●● ●
●● ●● ● ●● ● ●●
●● ●● ● ●●
● ● ● ● ● ● ● ●● ● ● ● ● ● ●
●● ● ● ● ● ●
● ● ● ● ●●
● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ●● ●
●● ●
●● ●● ●● ●● ●●● ●● ●●
●● ●● ● ●●
● ● ● ● ●● ●●● ●
● ●●
20000

● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ●● ● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ●● ● ●● ● ●●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●●● ● ●● ●● ●● ● ● ● ● ●● ● ● ● ● ●● ●● ●● ●
● ● ● ● ● ● ● ● ● ● ● ● ●●●● ●● ● ●● ●● ● ● ●●● ●●● ● ● ●
●● ●
●● ● ● ●
● ● ● ●●● ●●
● ● ● ● ● ● ● ●●●● ●● ● ●● ● ●● ●●
●●
●●
●●
●●
●● ●● ●● ●●
●●
●●
●● ●● ●● ●●
●● ●●
●●
● ●● ● ●● 70000
● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ● ●● ●● ● ●● ●● ●● ●
● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●● ● ● ●
●● ●
●● ●● ●
●● ● ●
●● ●●● ●● ● ●
●● ● ●
●● ●●● ●
●● ●
●● ● ●
●● ● ●
●● ● ● ●● ●●
● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ●● ●● ●● ● 60000
● ● ● ● ● ● ● ● ● ● ● ● ●●
●●
●●
●● ●● ● ●● ● ● ● ●● ●
●● ●
●● ●● ●● ●
●● ●
●● ●
●●
● ●● ● ●● ●
● ● ● ● ● ● ● ●● ● ● ● ● ●
●● ● ●
● ●● ●●● ● ●
●● ●
●● ●●● ● ●● ●
●
● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ● ●● ● ●● ●● ●● ●● ●● ●● ● ●●
● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●● ●●
●●
●●
●● ●● ● ●● ● ● ●
●● ●
●●
● ●● ●● ●
●● ●
●●
●●● 50000
● ● ● ● ● ● ● ● ● ● ● ● ●● ● ● ●● ● ● ●● ● ● ● ● ● ●● ●
● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ● ●● ●●● ●● ●● ●● ● ●●
10000

●● ●● ●● ●● ●● ●● ●● ●●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●
●●
●●
●●
●●
●● ●● ● ●● ● ● ● ●● ●
●● ●
●●● ● ● ● 40000
● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ● ●● ●● ● ● ● ● ●● ●●● ●●
● ● ● ● ● ● ● ●● ●● ●● ●● ●● ● ● ●● ●●
● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ●● ●●
●●
●●
●●
●●
●● ●● ● ●● ● ●●
● ●
●
● ●● 30000
● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●● ● ●● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ●● ●● ●
● ● ● ● ● ● ● ● ● ● ● ●
● ● ● ● ● ● ● ●●
● ●
●●
●●
●●
●● ●● ● ●● ● 20000
● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●● ● ●● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●● ●● ●●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●●
● ● ●● ● ●● ●
● ● ● ● ● ● ● ● ● ● ● ● ●●
● ● ● ●● ● 10000
● ● ● ● ● ● ●
● ● ● ● ● ● ● ● ● ● ● ● ● ● ●

0
● ● ● ● ● ● ● ● ● ● 0
0

0 10000 20000 30000 40000 50000 60000 70000

0 10000 20000 30000 40000 50000 60000

y[seq(1, n − (n%%3), 3)]
y[seq(1, n − (n%%2), 2)]

Marsaglia’s article Random Numbers fall mainly in the plane is a

pun on My Fair Lady refrain The Rain in Spain stays mainly in the
plain.
Roberto Innocente Random Numbers for Simulations
Multiply With Carry : MWC, Marsaglia

Concatenates 2 16-bit multiplies with carry (period ∼ 260 ) :

MWC:
Initial values: z0 = 362436069 , w0 = 521288629
zn ≡ 36969 ∗ (zn−1 &(216 − 1)) + zn−1 >> 16
wn ≡ 18000 ∗ (wn−1 &(216 − 1)) + wn−1 >> 16
output = (zn << 16) + wn

Roberto Innocente Random Numbers for Simulations

Add With Carry AWC , Subtract With Borrow SWB
(Marsaglia and Zaman 1991) I

AWC
x [ i ] = ( x [i - r ] + x [i - s ]+ c [i -1]) % m ;
c [ i ] = ( x [i - r ]+ x [i - s ]+ c [i -1]) / m ;

Initial state : S[0..k − 1] contains k initial integers (x0 , . . . , xk−1 )

and c = c0 , k = max(r , s). Today considered not good.

SWB
#define SWB (c++, bro=(x<y),t[c]=(x=t[UC (c+34)])-(y=t[UC (c+19)]+bro))

/* Global static v a r i a b l e s :*/

static UL z =362436069 , w =521288629 , jsr =123456789 ,
jcong =380116160;
static UL a =224466889 , b =7584631 , t [256] , x =0 , y =0 ,
bro ;
static unsigned char c =0;

Roberto Innocente Random Numbers for Simulations

Add With Carry AWC , Subtract With Borrow SWB
(Marsaglia and Zaman 1991) II

[13]

Roberto Innocente Random Numbers for Simulations

Knuth-TAOCP-2002

32-bit integer GFSR using lagged Fibonacci with subtraction,

F (100, 37, −). State 100 integers, 400 bytes. Period ∼ 2129
Knuth-TAOCP-2002
xj = (xj−100 − xj−37 ) (mod 230 )

Roberto Innocente Random Numbers for Simulations

Multiple Recursive Generator : MRG I

MRG:
xi = a1 xi−1 + . . . + ak xi−k (mod m) , i ≥ k

where m and k are positive integers called modulus and order and
the coefficients a1 , . . . , ak are in Zm . The state at step i is
si = (xi−k+1 , . . . , xi )T (a vector of length k). The initial state s0 is
required to be different from all 0. When m = p is a prime number
the ring Zp is a finite field and it is possible to choose the aj in
such a way that the period reaches ρ = p k − 1 (Knuth, 1998).
This maximal period is achieved iff the characteristic polynomial of
the recurrence P(z) = z k − a1 z k−1 − . . . − ak is a primitive
polynomial. Alanen and Knuth gave 3 conditions for verifying the
primitivity of P(z). In addition, a maximum-period MRG is known
to be equidistributed up to k-dimensions : every t-uple of Zp

Roberto Innocente Random Numbers for Simulations

Multiple Recursive Generator : MRG II

appears exactly p k−t times over the entire period p k − 1, except

the all-zeroes t-uple that apeears one time less.(See Niederreiter[6])

Roberto Innocente Random Numbers for Simulations

Matrix Congruential Generators I

An MRG can be implemented as a matrix multiplicative

congurential generator, which is a generator with state
St = Xt ∈ {0, . . . , m − 1}k for some modulus m and transition :

Xt = AXt−1 (mod m), t = 1, 2, . . .

The output is often taken to be :

Xt
Ut =
m
where A is an invertible k × k matrix and Xt is a k × 1 vector.

Roberto Innocente Random Numbers for Simulations

Number Theory (hors d'oevre) : Mersenne primes I

Mersenne numbers are those integers of the form 2k − 1.

Mersenne primes are those mersenne numbers that are primes. A
basic theorem says that if 2k − 1 is prime then also k is prime.
Since 1997 all newfound mersenne primes were discovered thru the
Great Internet Mersenne Prime Search ( https://en.wikipedia.
org/wiki/Great_Internet_Mersenne_Prime_Search) In
maxima you find some with (memory hog) :
for k :1 thru 20000 step 2 do
if primep ( k ) then if primep (2^ k -1) then print ( k ) ;

Roberto Innocente Random Numbers for Simulations

Number Theory (hors d'oevre) : Mersenne primes II

Roberto Innocente Random Numbers for Simulations

Number Theory (hors d'oevre) : Mersenne primes III

Recently Richard Brent ( [25] ), devised a new fast algorithm to

find primitive polynomials of Mersenne prime degree and reported
12 new found very large primitive trinomials over Z2 :

Roberto Innocente Random Numbers for Simulations

Number Theory : modular arithmetic :
ring Z/nZ and finite fields Zp I

Integers modulo m or congruence classes form a commutative ring.

If p is prime then Zp is a finite field also called Galois field GF.
If gcd(a, m) = 1 the least positive h for which ah ≡ 1 (mod m) is
called the multiplicative order of a modulo m.
if p is prime and gcd(g , p) = 1 and the multiplicative order of g
modulo p is (p − 1) then g is called a primitive root. ( p − 1 is the
max multiplicative order of g according to Fermat’s little theorem).
We said for every prime p, Zp is a finite field. Are those the only
finite fields ? No. All finite fields have cardinality p n and finite
fields with same cardinality are isomorphic. Exponents of the non
prime fields of order p n are the remainder classes of polynomials
over Zp (with coefficients in Zp ) modulus a monic irreducible
polynomial of degree n over Zp . For every n there is always at
least one. A notation for a generic finite field is Fpn or GF (p n ).

Roberto Innocente Random Numbers for Simulations

Number Theory : modular arithmetic :
ring Z/nZ and finite fields Zp II

Irreducible polynomials over finite fields Are those polynomials

that cannot be factored into non trivial polynomials over the same
field.
(Crandall, Pomerance, 2005) [7]
Theorem : If f (x) is a polynomial in Fp [x] of positive degree k,
the following statements are equivalent :
1 f (x) is irreducible;
2 gcd(f (x), x pj − x) = 1 for each j = 1, 2, . . . , bk/2c
k k/q
3 x p ≡ x (mod f (x)) and gcd(f (x), x p − x) = 1 for each
prime q | k.
Algorithm 2.2.9 (Crandall): is f (x) irreducible over Fp ?

Roberto Innocente Random Numbers for Simulations

Number Theory : modular arithmetic :
ring Z/nZ and finite fields Zp III
[ initialize ]
g(x) = x
[ Testing loop ]
for p :1 thru floor ( k /2) {
g ( x ) : g ( x ) ^ p mod f ( x ) ;
d ( x ) : gcd ( f ( x ) ,g ( x ) -x ) ;
if d ( x ) != 1) return ( NO ) ;
}
return ( YES ) ;

Primitive polynomials are those that have a root that is a

primitive root, that is, its powers generate all the elements of the
finite field. AN irreducible polynomial F (x) of degree m over
GF (p) where p is prime is a primitive polynomial if the smallest
integer such that F (x)|x n − 1 is n = p m − 1.
In the case of trinomials over GF (2) the test is simple. For every r
that is the exponent of a Mersenne prime 2r − 1 a trinomial of
degree r is primitive iff it is irreducible.
Roberto Innocente Random Numbers for Simulations
An escape : Maxima, package gf for finite fields
computations I

F.Caruso, et. al.Finite fields Computations in Maxima

gf_exp (a ,2) ;
gf_set_data (p , m ( x ) ) ;
make_list ( gf_random () ,i ,1 ,4) ;
gf_set_data (3 , x ^3+ x ^2+1) ;
mat : genmatrix ( lambda (\[ i , j \] ,
a :2* x ^3+ x ^2+1;
gen_random () ) ,3 ,3) ;
b : x ^2 -1;
gf_primitive () ;
gf_add (a , b ) ;
gf_index ( a ) ;
gf_mult (a , b ) ;
gf_p2n ( a ) ;
gf_inv ( b ) ;
gf_n2p () ;
gf_div (a , b ) ;
gf_logs (3) ;
gf_mul (a , gf_inv ( a ) ) ;
gf_powers (2) ;

Roberto Innocente Random Numbers for Simulations

Combined generators I

They were a great advance in the RNG arena. Here the heuristic is
that combining generators maybe of not so good quality for today
standard and shuffling, adding or selecting could make a better
generator. One class that was thoroughly studied was that of
Combined MRG. In some cases the theory can predict the period.
Methods used:
Add rn from 2 or more generators. If xi and yi are sequences
in [0..(m − 1)] then xi + yi (mod ()m) is also a sequence in
[0..(m − 1)].
XOR rn from 2 or more generators (Santa, Vazirani 1984)
Shuffle with a rn generator xi the output from another rn
generator yi (Marsaglia, Bray 1964) (e.g. keep last 100 items
from sequence yi use xi to choose from this buffer.

Roberto Innocente Random Numbers for Simulations

Combined LCG I
Proposition If the wi are L independent discrete rv such that wi is
uniform between 0 and d − 1:
1
P(wi = n) =
d
then
L
X
W = wi (mod d)
j=1

is uniform over 0 . . . (d − 1).

Proposition if we have a family of L generators where the
generator j has period pj and evolves according to the transition
function
sj,i = fj (sj,i−1 )
then the period of the sequence si = (s1,i , . . . , sL,i ) where
s0 = (s1,0 , . . . , sL,0 ) is a given seed is the least common multiple of
p1 , . . . , pL .
Roberto Innocente Random Numbers for Simulations
Combined MRG I

An MRG of order m is defined by :

xn = a1 xn−1 + . . . + ak xn−k

un = xn /m
wherem and k are positive integers and each ai belongs to Zm .
This recurrence has maximal period length mk − 1 attained iff m is
prime and the characteristic polynomial
P(z) = zk − a1 z k−1 . . . − ak is primitive. The last condition to
avoid too many computations can often be achieved with only 2
non zero coefficients like ar and ak with 1 <= r < k. If we have L
MRGs ∀l | 0 ≤ l < L − 1 :

xl,n = al,1 xl,n + . . . + al,k xl,n−k (mod ml )

Roberto Innocente Random Numbers for Simulations

Combined MRG II

with ml distinct primes and the recurrences have order k and

period mlk − 1, let dl be arbitrary integers each prime with ml for
each l, define :
L
X xl,n
wn = dl (mod 1)
ml
l=1

L
X
zn = dl xl,n (mod m1 )
l=1

un = zn /m1
then wn is exactly equivalent to an MRG with modulus
m = m1 m2 . . . mL . (L'Ecuyer 1998).

Roberto Innocente Random Numbers for Simulations

Wichman-Hill generator

This was one of the earliest combined generators. It combines 3

LCG.
Wichman-Hill
Xt = 171Xt−1 (mod m1 ) , (m1 = 30629)
Yt = 172Yt−1 (mod m2 ) , (m2 = 30307)
Zt = 170Zt−1 (mod m3 ) , (m3 = 30323)
Xt Yt Zt
Ut = + +
m1 m2 m3

The period of the triples (Xt , Yt , Zt ) is shown to be :

(m1 − 1)(m2 − 1)(m3 − 1) ∼ 6.95 × 1012
Performs well in tests, but the period is small.

Roberto Innocente Random Numbers for Simulations

L'Ecuyer MRG32k3a combined MRG I

A very famous combined MRG that was used extensively. Employs

2 MRG of order 3. The approximate period is 3 ∗ 1057 . It passes all
tests in TestU01. It is implemented in MATLAB, Mathematica,
IntelMKL library, SAS, etc.
MRG32k3a

Xt = (1403580∗Xt−2 −810728∗Xt−3 ) (mod m1 ), m1 = 232 −209

Yt = (527612∗Yt−1 −1370589∗Yt−3 ) (mod m2 ), m2 = 232 −22853

Xt − Yt + m1 Xt − Yt
Ut = if Xt ≤ Yt , if Xt > Yt
m1 + 1 m1 + 1

Roberto Innocente Random Numbers for Simulations

 L'Ecuyer MRG32k3a combined MRG II

# define norm 2 . 3 2 8 3 0 6 5 4 9 2 9 5 7 2 8 e -10 p1 = a12 * s11 - a13n * s10 ;

# define m1 4294967087.0 k = p1 / m1 ; p1 -= k * m1 ;
# define m2 4294944443.0 if ( p1 < 0.0) p1 += m1 ;
# define a12 1403580.0 s10 = s11 ; s11 = s12 ; s12 = p1 ;
# define a13n 810728.0 /* C o m p o n e n t 2 */
# define a21 527612.0 p2 = a21 * s22 - a23n * s20 ;
# define a23n 1370589.0 k = p2 / m2 ; p2 -= k * m2 ;
# define SEED 12345 if ( p2 < 0.0) p2 += m2 ;
static double s10 = SEED , s20 = s21 ; s21 = s22 ; s22 = p2 ;
s11 = SEED , s12 = SEED , s20 = /* C o m b i n a t i o n */
SEED , if ( p1 <= p2 )
s21 = SEED , s22 = SEED ; return (( p1 - p2 + m1 ) *
double MRG32k3a ( void ) norm ) ;
{ else
long k ; double p1 , p2 ; return (( p1 - p2 ) * norm ) ;
/* C o m p o n e n t 1 */ }

In MATLAB/Octave :

Roberto Innocente Random Numbers for Simulations

L'Ecuyer MRG32k3a combined MRG III

m1 =2^32 -209; M2 =2^32 -22853;

ax2p =1403580; ax3n =810728;
ay1p =527612; ay3n =1370589;

X =[12345 12345 12345] % initial X

Y =[12345 12345 12345] % initial Y

N =100; % compute N rn
U = zeros (1 , N ) ;
for t :1: N
Xt = mod ( ax2p * X (2) - ax3n * X (3) , m1 ) ;
Yt = mod ( ay1p * Y (1) - ay3n * Y (3) , m2 ) ;
if Xt <= Yt
U ( t ) =( Xt - Yt + m1 ) /( m1 +1) ;
else
U ( t ) =( Xt - Yt ) /( m1 +1) ;
end
X (2:3) = X (1:2) ; X (1) = Xt ; Y (2:3) = Y (1:2) ; Y (1) = Yt ;
end

Roberto Innocente Random Numbers for Simulations

Fourier DFT (spectral) test

The sequence of 0 and 1 is changed to -1 and 1

The DFT is applied to discover peaks in this sequence

Roberto Innocente Random Numbers for Simulations

Spectral test I

Knuth says: all good rng pass this test, all bad fail it : it is a very
important test. Usually the set of overlapping vectors :

Ls = {(xn , xn+1 , . . . , xn+s−1 ) | n ≥ 0}

is considered. This set exhibits a lattice structure for many

pseudorandom number generators such as LCG, multiple recursive,
lagged-Fibonacci, add-with-carry, subtract-with-borrow, combined
LCG, combined MRG. The test measures the maximal distance ds
between adjacent parallel hyperplanes that cover all vectors xn .
http://random.mat.sbg.ac.at/tests/theory/spectral/
An algorithm is based on the dual lattice derived from Ls . The
maximal distance is equal to one over the shortest vector in the
dual lattice.

Roberto Innocente Random Numbers for Simulations

Pierre L'Ecuyer

University of Montreal, Canada

developed together with R.Simard TestU01 : a
C library that performs empirical randomness
tests, 2007
developed the famous combined generator
MRG32k3a
developed with F.O. Panneton and
M.Matsumoto
WELL (Well Equidistributed Long-period Linear
rng : one of the rising stars among rng)

Roberto Innocente Random Numbers for Simulations

Linear Feedback Shift Register LFSR I

Differently from the others this is a random bit generator and

not a random integer or float generator. The theory of this
sequence generator has its roots in error correcting codes and
cryptography (in particular streaming ciphers). It was devised
thinking about an easy hardware implemention of it so that it can
be very fast and efficient.
Golomb [9] is the standard reference for this generator.
It can easily be implemented in hardware as a sequence of flip-flops
that at every clock push their content to the element on the right.
An input is provided by a feedback connection based on a linear
function (usually an XOR that on F2 is the same as an add
operation) on some bits of the register (the rightmost bit should be
used , otherwise the LFSR is called singular and is not of interest).

Roberto Innocente Random Numbers for Simulations

Linear Feedback Shift Register LFSR II
How you indicate a LFSR ?
Shift Register With LFSR(L,poly). Eg.
Serial
Input
1 0 1 1 0
Serial
Output
LFSR(4, 1 + x + x 4 )
Linear Feedback Shift Register LFSR
Theorem : Let P(x) be a
f()
connection polynomial of degree
……. L over F2 [x]:
Serial
1 0 1 1 0
Output If P(x) is irreducible over
x x^2 x^3 x^4 x^5
F2 then for each nonzero
A common function used for the feedback is the
adddition in the finite field F2 (= XOR) between some
of the bits in the register (called taps).
seed produces an output
The LFSR is connected with a polynomial in F2 with all
the powers of x xored for the feedback. For instance sequence with period the
XOR XOR least N such that P(x)
divides 1 + x N .
Serial
1 0 1 1 0
Output

x x^2 x^3 x^4 x^5

If P(x) is a primitive
3 5
P ( x)=1+ x + x , LFSR(5,1+ x + x )
3 5
polynomial then each seed
produces an output
sequence of maximal
length 2L − 1.
Roberto Innocente Random Numbers for Simulations
Primitive polynomials over F2 I

Irreducible/Primitive
trinomials
x m + x k + 1 over F2 .
Handbook of Applied
Cryptography,
Menezes ( [10] )

Roberto Innocente Random Numbers for Simulations

RNG based on Linear recurrences on F2 I
General framework introduced by Niederreiter and then L'Ecuyer.
It comprises most of the methods. F2 is the finite field with two
elements, 0 and 1. General framework :
xi = Axi−1 (transition function)
yi = Bxi (output function)
ui = yi,l−1 2−l = .yi,0 yi,1 yi,2 . . .
P

where xi = (xi,0 , xi,1 , . . . , xi,k−1 )T ∈ Fk2 and

yi = (yi,0 , yi,1 , . . . , yi,w −1 )T ∈ Fw
2 . A is a k × k transformation
matrix and B a w × k output matrix.
The characteristic polynomial of the matrix A can be written:

P(z) = det(A − zI) = z k − a1 z k−1 − a2 z k−2 − . . . − ak−1 z − ak

Both xi and yi obey the same recurrence

xi,j = a1 xi−1,j + . . . + ak xi−k,j mod2

Roberto Innocente Random Numbers for Simulations
RNG based on Linear recurrences on F2 II

The period of this sequence is full = 2k − 1 if P(z) is a primitive

polynomial over F2 .
More the matrices are sparse, more efficient is the computation to
get a rn, but this against the fact that if the matrix don’t mix
enough the state, the resultant generator will have poor statistical
qualities.

Roberto Innocente Random Numbers for Simulations

Generalized feedback shift register GFSR, (Lewis and
Payne 1973) I

There are two ways to get random integers or floats from an LFSR
random bit generator:
run an LFSR generator l times and get l bits from it and
consider them as the binary fraction of a float in (0, 1) or an
integer [0..(2l − 1)].
x = (b0 , . . . , bl−1 )T produced by a LFSR ( usually based on a
trinomial)
bi = bi−p ⊕ bi−p+q
are taken to represent the fraction of a float or an integer with
l bits ( l-wise decimation of the sequence of bi ). If l is
relatively prime with 2p − 1 (the period of the LFSR), also the

Roberto Innocente Random Numbers for Simulations

Generalized feedback shift register GFSR, (Lewis and
Payne 1973) II

period of the l-tuples will be 2p − 1. The blocks xi satisfy the

same recurrence of the sequence of bits bi :

xi = xi−p ⊕ xi−p+q

and are connected with the trinomial

xp + xr + 1

where r = p − q. The initial state is a sequence of p bits.

Roberto Innocente Random Numbers for Simulations

Generalized feedback shift register GFSR, (Lewis and
Payne 1973) III

Tausworthe (1965)
An RNG built on this is sometimes indicated as R(r, p)

xn = 0.bnl bnl+1 bnl+2 . . . bnl+(l−1)

l−1
X
Un = bnl+j 2−j−1
j=0

This method is, of course, inefficient.

Roberto Innocente Random Numbers for Simulations

Generalized feedback shift register GFSR, (Lewis and
Payne 1973) IV

A better way is to use the LFSR in parallel over a word of w

bits. In this case the seed is made of p words. It generates
random integers in 0..(2w − 1). ( [26] )

GFSR (Lewis, Payne) :

xn = xn−p ⊕ xn−p+q

It has period (2p − 1). The initial values are p integers. See
next picture.

Roberto Innocente Random Numbers for Simulations

Generalized feedback shift register GFSR, (Lewis and
Payne 1973) V

Roberto Innocente Random Numbers for Simulations

Twisted generalized feedback shift register TGFSR

Matsumoto [27]
Pros of GFSR : fast generation of rn, sequence has arbitrarily long
period, implementation does not depend on word size.
Cons : selection of seeds its critical and good initialization is time
consuming, period 2p − 1 is quite smaller than the storage area
would allow.

xl+n = xl+m ⊕ xl A , (l = 0, 1, . . .)
where A is a w × w matrix with 0, 1 components. With suitable
choices of n, m, A the TGFSR generator attains the maximal
period of 2nw − 1. Because it has maximal period it is
n − equidistributed. The trick is simply to update xl with a twist :

xl = (xl+m (mod n)) ⊕ shiftright(xl ) ⊕ (0 if LSB(xl = 0) else a)

Roberto Innocente Random Numbers for Simulations

Parallel random numbers I

From a rn generator x = (xn ) , we can easily obtain parallel

streams :
partitioning in j lagged subsequences :
(j)
ωk = (xkn+j )m≥0 , k ≥ 2 , 0 ≤ j < k (aka leapfrog)
partitioning in consecutive blocks of length l :
(k)
ψl = (xkl+n )l−1n=0 , k ≥ 0 (aka sequence splitting)
previous methods are good if the generator allows easily to
skip-ahead like LCGs. It is impractical for LFSR generators.
For these you run the same LFSR with different initial seeds
on every processor. The initialization of the seeds tables can
be done with an LCG for instance.

Roberto Innocente Random Numbers for Simulations

SPRNG : Scalable Parallel
Random Number Generator library I

Developed at Florida State University (current version in

C++/Fortran is 5.0) from 1999 to today by M.Mascagni et al.
Download it from
http://www.sprng.org/Version5.0/sprng5.tar.bz2 and
tar xjf sprng5 . tar . bz2
cd sprng5
./ configure
make
cd check
./ checksprng
./ timesprng

Based on the 5 generators :

1 Combined Multiple Recursive Generator (MRG)
zn = xn + yn ∗ 232 mod 264
yn = 107374182 ∗ yn−1 + 104480 ∗ yn−5 mod 2147483647
and x is the sequence produced by the 64 bit LCG.
Roberto Innocente Random Numbers for Simulations
SPRNG : Scalable Parallel
Random Number Generator library II
2 48 bit LCG with Prime Addend
xn = axn−1 + p mod M , M = 248
3 64 bit LCG with Prime Addend
4 Modified Lagged Fibonacci Generator
zn = xn XOR yn
where XOR is the exclusive-or operator and x and y are
sequences obtained from Lagged Fibonacci sequences of the
following form:
xn = xn−k + xn−l mod M
yn = yn−k + yn−l mod M
5 Multiplicative Lagged Fibonacci Generator
xn = xn−k ∗ xn−l mod M
6 Prime Modulus LCG
xn = a ∗ xn−1 mod (261 − 1) where the multiplier a differs for
each stream
How to use it ?
In a serial program :
Roberto Innocente Random Numbers for Simulations
SPRNG : Scalable Parallel
Random Number Generator library III

1 define the macro SPRNG_DEFAULT to use the simple interface. In

C #define SPRNG_DEFAULT 0
2 C users should include sprng_cpp.h, Fortran users sprng_f.h
3 If the user wants he can call an initialization function
init_sprng. This function has 4 parameters:
stream number, total number of generators, seed, multiplier.
4 calling now sprng() will provide a double precision number in
(0, 1)
In a parallel program :
1 define the macro SPRNG_DEFAULT to use the simple interface
#define SPRNG_DEFAULT 0
2 define the macro USE_MPI to instruct the generator to use MPI
during initialization
3 C users should include sprng_cpp.h, Fortran users sprng_f.h
4 before calling any sprng function the user should call
MPI_Init

Roberto Innocente Random Numbers for Simulations

Salmon,etc. Shaw Research(Anton) : Parallel Random
Numbers: As easy as 1,2,3 I

Counter based random numbers.

xn = bk (n)

Inherently parallel because there is no dependence between

successive xn in the sequence. bk is a keyed bijection with key k.
They start from well known cryptographic ciperhs that implement
a keyed bijection : AES , Threefish. These are too slow for
simulations. Then they try to change the algorithms reducing the
cryptographic strength and complexity. What comes out they
called: ARS, Threefry and Philox and they report very good
performance for them.

Roberto Innocente Random Numbers for Simulations

Statistical Tests I

First battery of tests described by Knuth in TAOCP in 1969,

but no implementation was made available for them
diehard tests by Marsaglia, 1996
https://wayback.archive.org/web/20160125103112/http://stat.fsu.edu/
pub/diehard/
license status of these tests is not clear
TestU01 by P.L'Ecuyer and R.Simard, implements most of
Knuth tests. Code copyrighted by P.L'Ecuyer. 2007.
STS Statistical Test Suite for Random and Pseudorandom
Number Generators for Cryptography (National Institute of
Standards and Technology), 2001/2010. Consist of 16 tests.
In the public domain because made by a gov agency but for
the algorithms this is not clear

Roberto Innocente Random Numbers for Simulations

Statistical Tests II

DieHarder Robert G.Brown parameterized all diehard tests

and put them under GPL. In addition he included some
re-programmed STS tests. The declared aim is to add all STS
tests and put them under GPL.

Roberto Innocente Random Numbers for Simulations

Test of Hypothesis, p-Values I

1 Make an initial assumption that is usually called H0 (null

hypothesis). An alternative is called alternative hypothesis
and is indicated by Ha
2 Collect evidence
3 Based on data collected reject or not reject the assumption
(according if H0 is unlikely or likely)
In the p-value approach, in the case H0 is true, the probability of
observing a more extreme statistic is computed. If the p-value is
small, say less than α then it is unlikely, if the p-value is large it is
likely and the null hypothesis is not rejected.
In most of the tests for rng H0 is the hypothesis that the random
numbers are uniform : ∼ U (0, 1). This is equivalent to say that
for any t > 0 the t-uple (u0 , u1 , . . . , ut−1 ) is uniformly distributed
over [0, 1]t . We have two kind of statistical test:

Roberto Innocente Random Numbers for Simulations

Test of Hypothesis, p-Values II

Single level tests

It observes the value of a statistic Y , say y and rejects H0 if
the p-value
p = P[Y ≤ y | H0 ]
or
p = P[Y ≥ y | H0 ]
that is, the probability that an outcome, given H0 , is more
extreme than the value measured is too much close to 0 or 1.
Two level tests
In a second order test one generates N independent copies of
Y say Y1 , Y2 , . . . , YN replicating the first order test.

Roberto Innocente Random Numbers for Simulations

Test of Hypothesis, p-Values III

Usually for most of the test the distribution of Y is either χ2 ,

normal or Poisson. In these 3 case their sum has the same
distribution. That is if Y is χ2 with k degrees of freedom then Y e
2 2
is χ with Nk degrees of freedom. (χ is the distribution of the
sum of normal variates).
if Y is Poisson with k degrees of freedom and mean λ, then Y e is
Poisson with Nk deg of freedom and mean λ, if Y is normal with
mean µ and variance σ 2 then Y e is normal with mean Nµ and
2 2
variance N σ , etc.
E.g. If the statistic Y ∼ N (1/2, 1/sqrt(12 ∗ N)) should be
normally distributed around 1/2 with sd = 1/sqrt(12)/sqrt(N) and
we measure 0.45 then

Roberto Innocente Random Numbers for Simulations

Test of Hypothesis, p-Values IV

Roberto Innocente Random Numbers for Simulations

Test of Hypothesis, p-Values V

Code in R :
# hypothesis testing
xmi =0.2; xma =0.8; xl =0.45
x = seq ( xmi , xma ,0.01)
xmean =1/2;
N =30
sigma =1/ sqrt (12) ; stdev = sigma / sqrt ( N )
plot (x , dnorm (x , m = xmean , sd = stdev ) , type = ’n ’)
title ( ’ Test of hypothesis : H0 ~ N (1/2 , sd = sigma / sqrt ( N )
) ’)
str = bquote ( H_0 ~ N (1/2 ,1/ sqrt (12*50) ) )
lines (x , dnorm (x , m = xmean , sd = stdev ) , col = ’ red ’)
text (0.5 ,6.5 , ’ H0 ~ N (1/2 , sd = sigma / sqrt ( N ) ) ’, col = ’ red ’)
xx = c ( xmi -0.01 , seq ( xmi , xl ,0.01) , xl )
yy = c (0 , dnorm ( seq ( xmi , xl ,0.01) ,m = xmean , sd = stdev ) ,0)
polygon ( xx , yy , col = ’ skyblue ’)

Roberto Innocente Random Numbers for Simulations

Uniformity or goodness-of-fit tests I

Let H0 be the hypothesis that our RNG produces samples

∼ U (0, 1). For the continuous uniform ditribution wepknow that
the mean µ = 1/2 and the standard deviation is s = 1/12. Let
us produce some hundredths samples of some size n > 30and take
their mean. According to the Central Limit Theorem the sample
mean is distributed like a normal distribuition

Roberto Innocente Random Numbers for Simulations

Uniformity or goodness-of-fit tests II

N (µ, sd =
p
1/12/sqrt(n)).

Roberto Innocente Random Numbers for Simulations

Uniformity or goodness-of-fit tests III
# R code
# central limit theorem
# average distr is normal ( mu , sd / sqrt ( n ) )
nlim =1500; rlim =600
x = seq (0.4 ,0.6 ,.0005)
# m [] is an array or nlim means
m < - numeric ( nlim )
for ( i in (1: nlim ) ) { m [ i ]= mean ( runif ( rlim ) ) }
png ( ’ clt -1. png ’)
hist (m , xlim = c (0.4 ,0.6) , ylim = c (0 ,50) , freq = FALSE ,
main = ’ Histogram of m vs . Normal pdf ’)
lines (x , dnorm (x , m =1/2 , sd = sqrt (1/12) * sqrt (1/ rlim ) ) )
# pause here
png ( ’ clt -2. png ’)
plot . new ()
plot ( ecdf ( m ) , main = NULL , col = ’ red ’)
title ( main = ’ Empirical CDF of m vs Normal CDF ’)
lines (x , pnorm (x , m =1/2 , sd = sqrt (1/12) * sqrt (1/ rlim ) ) )

ks . test (m , pnorm , mean =1/2 , sd =1/ sqrt (12) / sqrt ( rlim )

, alternative = c (" less ") )
# E . G .: One - sample Kolmogorov - Smirnov test
# data : m

Roberto Innocente Random Numbers for Simulations

Uniformity or goodness-of-fit tests IV

# D ^ - = 0.00813 , p - value = 0.8201

# alternative hypothesis : the CDF of x lies below the null
hypothesis

Kolmogorov-Smirnov K-S test

χ2 test

Roberto Innocente Random Numbers for Simulations

Using R: Kolmogorov-Smirnov I

Kolmogorov-Smirnov measures the maximal deviations between

the expected theoretical cumulative distribution function (CDF)

Roberto Innocente Random Numbers for Simulations

Using R: Kolmogorov-Smirnov II

and the empirical distribution function (ECDF) obtained from

the data.

Roberto Innocente Random Numbers for Simulations

Using R: Kolmogorov-Smirnov III

How to compute a p-value from k-s test?

The exact formula for the distribution is :

  √ √
s X N Ns − k
k N − k
N−k−1
P(kN± ≤ s) = √ (−1) k
1+
N √ k N N
0≤k≤b Nsc

For N ≥ 100 we can use :

−2(s+ 1
√ )2
P(kN± ≤ s) = 1 − e 6 N

Roberto Innocente Random Numbers for Simulations

TestU01 , P.L'Ecuyer and R.Simard I

TestU01 contains over 200 predefined rng for test purposes (LCG, MRG,
combined MRG, lagged-Fibonacci, AWC, SWB, MWC LFSR, combined
LFSR, GFSR, twisted GFSR, Mersenne twisters, WELL, . . . ). It divides
its tests for U(0, 1) variates in 3 batteries (SmallCrush, Crush[96 tests],
BigCrush[106 tests]) of increasing time and complexity:
e.g. exec time respectively 14 sec(SmallCrash), 1 hour(Crush), 5.5
hours(BigCrush).
Small Crush battery of tests :
1 smrsa BirthdaySpacings 6 sknuth MaxOft
2 sknuth Collision 7 svaria WeightDistrib
3 sknuth Gap 8 smarsa MatrixRank
4 sknuth SimpPoker 9 sstring HammingIndep
5 sknuth CouponCollector swalk RandomWalk1
10

It’s not so friendly to use as dieharder. It requires you to write

some code. E.g. :

Roberto Innocente Random Numbers for Simulations

TestU01 , P.L'Ecuyer and R.Simard II

# include " ulcg . h "

# include " unif01 . h "
# include " bbattery . h "

int main ( void )

{
unif01_Gen * gen ;
gen = ul cg_Crea teLCG (2147483647 , 16807 , 0 , 12345) ;
b b a t t e r y _ S m a l l C r u s h ( gen ) ;
ulcg_DeleteG en ( gen ) ;
return 0;
}

generates an LCG and submits it to the SmallCrush battery of

tests. To compile and link it :
gcc bat1.c -o bat1 -ltestu01 -lprobdist -lmylib -lm

Roberto Innocente Random Numbers for Simulations

DieHarder R.G.Brown I

80 generators (diehard + NIST/STS + RGBrown + Knuth), 31

tests. Revised version :
https://github.com/seehuhn/dieharder In Ubuntu simply
apt-get install dieharder.

dieharder -g -1 : list internal generators available ( 80)

dieharder -l : list available tests ( 31)
dieharder -a -g 13 : apply all tests to the generator 13 =
mt19937
produce numbers with random_write and then analyze with
cat fn |dieharder -a -g 201A

cat /dev/urandom | dieharder -g 200 -a

The version that is now an Ubuntu package is very user-friendly

and it allows to pipe rn to it or simply to give it a file of rn
(formatted or not) to digest.

Roberto Innocente Random Numbers for Simulations

NIST Statistical Test Suite - tests I

http://csrc.nist.gov/groups/ST/toolkit/rng/documentation_software.html
Usage : ./assess <length_of_rn>

Roberto Innocente Random Numbers for Simulations

Primitive polynomials of Mersenne prime order I

Primitive polynomials over F2 of

degree m where 2m − 1 is a mersenne
prime. Trinomials/pentanomials such
that x m + x k + 1 is irreducible over
F2 .

Roberto Innocente Random Numbers for Simulations

Mersenne Twister MT I
Many different versions exist. Mostly used is the revised MT19937
in C : http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/
MT2002/emt19937ar.html. It is a Twisted GFSR. Originally
developed by Matsumoto and Nishimura (1998), period 219937 − 1.

MT (linear recursion over F2 ) :

xk+n = xk+m ⊕ ((xku || xk+l

l
)A)

n is the degree of recursion

A is a w × w matrix chosen to make simple the multiplication
 
0 1 0 0 ... 0
 0 0 1 0 ... 0
 
 .. .. .. .. .. 
 . . . . . 0
 
 0 0 0 0 ... 1
aw −1 aw −2 aw −3 aw −4 . . . a0
Roberto Innocente Random Numbers for Simulations
Mersenne Twister MT II
k is 0, 1, 2, . . .
x0 , x1 . . . , xn−1 are initial seeds
m is a middle index 1 ≤ m ≤ n
l
xk+1 are lower or rightmost r bits of xk+1
xku are upper or leftmost w − r bits of xk
⊕ is bitwise XOR
Illustration of state transition (from Jagannatam):

Parameter Value
n 624
w 32
r 31
m 397

Roberto Innocente Random Numbers for Simulations

Mersenne Twister MT III

Roberto Innocente Random Numbers for Simulations

Mersenne twister : MT variants I

TinyMT (127 bits of state), 2 versions tinymt32 and tinymt64,

first outputs 32 bits unsigned integers or single floats, second 64
bits unsigned integers or double precision floats, period only 2127−1
but very small state. State initialized by the function TinyMTDC.
Authors: M.Saito,M.Matsumoto
MTGP MT for Graphic Processors. (Saito, Matsumoto) 32 or 64
bit integers as output or single precision or double precision floats.
For the 32 bit periods of 211213 − 1, 223209 − 1, 244497 − 1. Cuda
and OpenCL code. To provide the many parameters used there is
and MTGP Dynamic Creator routine (MTGPDC.
SFMT SIMD-oriented Fast MT.(2006 Saito, Matsumoto) is a
LFSR that generates 128 bit RN integer at each step. It uses 128
bit integers supported on modern CPU and SIMD. Source in
standard C, SSE@ isntructions + C, AltiVec instructions + C.
Periods 2607 − 1, 2216091 − 1.

Roberto Innocente Random Numbers for Simulations

xorshift (Marsaglia 2003)

Period 2k − 1 with k = 32, 64, 96, 128, 160, 192. Produces integers
∈ [0 . . . 232 − 1], by means of the XOR instruction. In C : y^(y<<a),
in Fortran : IEOR(y,ishft(y,a)). To give an idea of the power of this
procedure, given 4 32 bits seeds x, y , z, w the sequence :
tmp =( x ^( x < <15) ) ;
x=y;y=z;z=w;
return w =( w ^( w > >21) ) ^( tmp ^( tmp > >4) ) ;

provides 2128 − 1 random 32-bits integers that survive the diehard

battery. We have a seed set ∈ Zm made up of m-tuples
z = (x1 , x2 , . . . , xm ) and a one-to-one function f () on Zm . The
output of the rng is f (z), f 2 (z), f 3 (z), . . ..

Roberto Innocente Random Numbers for Simulations

Numerical Recipes RNG ! I

The authors(Chapter 7 has 100 pages about Random Numbers)

claim this to be the ultimate RNG : ”this is our
suspenders-and-belt, full-body-armor, never any-doubt generator”,
but this time they don’t offer any reward to someone discovering
that it is bad. In the first edition of their book both ran0() and
ran1() where flawed.
This to say that the field is mined and even very skilled
people can easily stumble.
struct Ran
{ Ullong u ,v , w ;
Ran ( Ullong j ) : v ( 4 1 0 1 8 4 2 8 8 7 6 5 5 1 0 2 0 1 7 LL ) ,w (1) {
u = j ^ v ; int64 () ; v = u ; int64 () ; w = v ; int64 () ;}
inline Ullong int64 ()
{ u = u * 2 8 6 2 9 3 3 5 5 5 7 7 7 9 4 1 7 5 7 LL + 7 0 4 6 0 2 9 2 5 4 3 8 6 3 5 3 0 8 LL ;
v ^= v > >17; v &= v < <31; v ^= v > >8;
w = 4294957665 U *( w &0 xffffffff ) +( w > >32) ;
Ullong x = u ^( u < <21) ; x ^= x > >35; x ^= x < <4; return ( x + v ) ^ w ;
}

Roberto Innocente Random Numbers for Simulations

Numerical Recipes RNG ! II

inline Doub doub () { return 5 . 4 2 1 0 1 0 8 6 2 4 2 7 5 2 2 1 7 E ^20* int64 ()

;}
inline Uint int32 () { return ( Uint ) int64 () ;}
};

Roberto Innocente Random Numbers for Simulations

P.L' Ecuyer, R.Simard: RngStreams I

An object oriented Random-Number Generator. In C, C++, Java :

http://www.iro.umontreal.ca/~lecuyer/myftp/streams00/
http://statmath.wu.ac.at/software/RngStreams/
To install it :
wget http :// statmath . wu . ac . at / software / RngStreams / rngstreams
-1.0.1. tar . gz
tar xz rngstreams -1.0.1. tar . gz
cd rngstreams -1.0.1
./ configure -- prefix =/ usr / local
make
make install
make check

Roberto Innocente Random Numbers for Simulations

Panneton, L'Ecuyer, Matsumoto 2006 : WELL

Well Equidistributed Long-period Linear , 2006

It is based on linear recurrences modulo 2 over F2
Panneton, F. O.; l’Ecuyer, P.; Matsumoto, M. (2006).
”Improved long-period generators based on linear recurrences
modulo 2” (PDF).
ACM Transactions on Mathematical Software
http://www.iro.umontreal.ca/~panneton/WELLRNG.html
512, 1024, 19937, 44497 bits implementations readily available.
Their implementations makes a TGFSR become ME (Maximally
Equidistributed) adding at the output a Matsumoto-Kurita
tempering of the output (TTGFSR : Tempered Twisted GFSR ).
Period of WELL19937 is 219937 − 1.

Roberto Innocente Random Numbers for Simulations

Minimal Standard MINSTD Park-Miller generator I
75 = 16807, 231 − 1 = 2147483647
MINSTD
xn ≡ 16807 ∗ xn−1 (mod 231 − 1) , LCG (75 , 0, 231 − 1)

Stephen K. Park; Keith W. Miller (1988).

Random Number Generators: Good Ones Are Hard To Find
Given the dynamic nature of the area, it is difficult for
nonspecialists to make decisions about what generator to
use. ”Give me something I can understand, implement
and port... it needn’t be state-of-the-art, just make sure
it’s reasonably good and efficient.” Our article and the
associated minimal standard generator was an attempt to
respond to this request. Five years later, we see no need
to alter our response other than to suggest the use of the
multiplier a = 48271 in place of 16807.

Roberto Innocente Random Numbers for Simulations

Bad news :
most rng provided by languages and libraries are bad !

These were bad at a certain point in time :

C-library rand(), random() and drand48()
Java.util.Random
standard Perl rand
Python random()
Matlab’s rand
Mathematica SWB generator
ran0() and ran1() in the original Numerical recipes book

Roberto Innocente Random Numbers for Simulations

Good news: good rng can be simple I

KISS99 (proposed by G.Marsaglia) period ∼ 1043 :

static unsigned int x = 123456789 , y = 362436000 ,
z = 521288629 , c = 7654321;
unsigned int kiss ()
{
unsigned long long t , a = 698769069 ULL ;
x = 69069* x +12345;
y ^= (y < <13) ; y ^= (y > >17) ; y ^= (y < <5) ;
t = a * z + c ; c = (t > >32) ;
return x + y +( z = t ) ;
}

In MATLAB/Octave :

Roberto Innocente Random Numbers for Simulations

Good news: good rng can be simple II
% seeds : correct variable types are crucial
A = uint32 (12345) ; B = uint32 (65435) ; Y =12345; Z = uint32 (34221) ;
N =100; % compute N rn
U = zeros (1 , N ) ;
for t =1: N
% 2 MWC generators
A =36969* bitand (A , uint32 (65535) ) + bitshift (A , -16) ;
B =18000* bitand (B , uint32 (65535) ) + bitshift (B , -16) ;
% MWC : A and B are low and high 16 bits
X = bitshift (A ,16) + B ;
% CONG : LCG
Y = mod (69069* Y +1234567 ,4294967296) ;
% SHR3 : 2 - shift register generator
Z = bitxor (Z , bitshift (Z ,17) ) ;
Z = bitxor (Z , bitshift (Z , -13) ) ;
Z = bitxor (Z , bitshift (Z ,5) ) ;
% combine to form KISS99
KISS = mod ( double ( bitxor (X , uint32 ( Y ) ) ) + double ( Z )
,4294967296) ;
U ( t ) = KISS /4294967296;
end
U (100)

Roberto Innocente Random Numbers for Simulations

Analysis of KISS99 I

A maxima script to show the working of KISS99 :

n :32;
u :[ b31 , b30 , b29 , b28 , b27 , b26 , b25 , b24 , b23 , b22 , b21 , b20 ,
b19 , b18 , b17 , b16 , b15 , b14 , b13 , b12 , b11 , b10 ,
b09 , b08 , b07 , b06 , b05 , b04 , b03 , b02 , b01 , b00 ];
y : transpose ( u ) ;
/* shif left */
shl : genmatrix ( lambda ([ i , j ] , if i = (j -1) then 1 else 0) ,n , n ) ;
/* shif right */
shr : genmatrix ( lambda ([ i , j ] , if i = ( j +1) then 1 else 0) ,n , n ) ;

/* shif right k places */

shrn ( k ) := shr ^^ k ;
/* shif left k places */
shln ( k ) := shl ^^ k ;
/* In GF (2) XOR is the same as + */
y : y + shln (13) . y ;
y : y + shrn (17) . y ;
y : y + shln (5) . y ;

Roberto Innocente Random Numbers for Simulations

Analysis of KISS99 II
[ b31 + b26 + b18 + b13 ]
[ b30 + b25 + b17 + b12 ]
[ b29 + b24 + b16 + b11 ]
[ b28 + b23 + b15 + b10 ]
[ b27 + b22 + b14 + b09 ]
[ b26 + b21 + b13 + b08 ]
[ b25 + b20 + b12 + b07 ]
[ b24 + b19 + b11 + b06 ]
[ b23 + b18 + b10 + b05 ]
[ b22 + b17 + b09 + b04 ]
[ b21 + b16 + b08 + b03 ]
[ b20 + b15 + b07 + b02 ]
[ b31 + b19 + b18 + b14 + b06 + b01 ]
[ b30 + b18 + b17 + b13 + b05 + b00 ]
[ b29 + b17 + b16 + b12 + b04 ]
[ b28 + b16 + b15 + b11 + b03 ]
[ b27 + b15 + b14 + b10 + b02 ]
[ b31 + b26 + b18 + b14 + b13 + b09 + b01 ]
[ b30 + b25 + b17 + b13 + b12 + b08 + b00 ]
[ b29 + b24 + b16 + b12 + b11 + b07 ]
[ b28 + b23 + b15 + b11 + b10 + b06 ]
[ b27 + b22 + b14 + b10 + b09 + b05 ]
[ b26 + b21 + b13 + b09 + b08 + b04 ]

Roberto Innocente Random Numbers for Simulations

Analysis of KISS99 III

[ b25 + b20 + b12 + b08 + b07 + b03 ]

[ b24 + b19 + b11 + b07 + b06 + b02 ]
[ b23 + b18 + b10 + b06 + b05 + b01 ]
[ b22 + b17 + b09 + b05 + b04 + b00 ]
[ b21 + b08 + b04 ]
[ b20 + b07 + b03 ]
[ b19 + b06 + b02 ]
[ b18 + b05 + b01 ]
[ b17 + b04 + b00 ]

Roberto Innocente Random Numbers for Simulations

KISS93 and other 6 random number gen by G.Marsaglia I

Main program in C :
# include < stdio .h >
int main ( void ) {
int i ; UL k ;
settable (12345 ,65435 ,34221 ,12345 ,9983651 ,95746118) ;

for ( i =1; i <1000001; i ++) { k = LFIB4 ;} printf ( " % u \ n " ,k -1064612766 U )

;
for ( i =1; i <1000001; i ++) { k = SWB ;} printf ( " % u \ n " ,k - 627749721 U ) ;
for ( i =1; i <1000001; i ++) { k = KISS ;} printf ( " % u \ n " ,k -1372460312 U ) ;
for ( i =1; i <1000001; i ++) { k = CONG ;} printf ( " % u \ n " ,k -1529210297 U ) ;
for ( i =1; i <1000001; i ++) { k = SHR3 ;} printf ( " % u \ n " ,k -2642725982 U ) ;
for ( i =1; i <1000001; i ++) { k = MWC ;} printf ( " % u \ n " ,k - 904977562 U ) ;
for ( i =1; i <1000001; i ++) { k = FIB ;} printf ( " % u \ n " ,k -3519793928 U ) ;
}

The random number generators using macro definitions :

Roberto Innocente Random Numbers for Simulations

KISS93 and other 6 random number gen by G.Marsaglia II
# define znew ( z =36969*( z &65535) +( z > >16) )
# define wnew ( w =18000*( w &65535) +( w > >16) )
# define MWC (( znew < <16) + wnew )
# define SHR3 ( jsr ^=( jsr < <17) , jsr ^=( jsr > >13) , jsr ^=( jsr < <5) )
# define CONG ( jcong =69069* jcong +1234567)
# define FIB (( b = a + b ) ,( a =b - a ) )
# define KISS (( MWC ^ CONG ) + SHR3 )
# define LFIB4 ( c ++ , t [ c ]= t [ c ]+ t [ UC ( c +58) ]+ t [ UC ( c +119) ]+ t [ UC ( c
+178) ])
# define SWB ( c ++ , bro =( x < y ) ,t [ c ]=( x = t [ UC ( c +34) ]) -( y = t [ UC ( c
+19) ]+ bro ) )
# define UNI ( KISS *2.328306 e -10)
# define VNI (( long ) KISS ) *4.656613 e -10
# define UC ( unsigned char ) /* a cast o p e r a t i o n */
typedef unsigned long UL ;

/* Global static v a r i a b l e s : */
static UL z =362436069 , w =521288629 , jsr =123456789 , jcong
=380116160;
static UL a =224466889 , b =7584631 , t [256] , x =0 , y =0 , bro ;
static unsigned char c =0;

/* Example p r o c e d u r e to set the table , using KISS : */

Roberto Innocente Random Numbers for Simulations

KISS93 and other 6 random number gen by G.Marsaglia III

void settable ( UL i1 , UL i2 , UL i3 , UL i4 , UL i5 , UL i6 )
{ int i ; z = i1 ; w = i2 ; jsr = i3 ; jcong = i4 ; a = i5 ; b = i6 ;
for ( i =0; i <256; i = i +1) t [ i ]= KISS ; }

Roberto Innocente Random Numbers for Simulations

KISS diagram

Roberto Innocente Random Numbers for Simulations

Ferrenberg: 2D Ising model montecarlo failure : CONG, 2
SHR, SWC, SWCW I
A word of caution about use of RNG in large simulations. These
generators that performed relatively well on normal tests (at that
time : 1992) were reported to fail a Monte Carlo test of the 2D
Ising model [20]. Some then proposed to use simulation with a
theoretical predictable outcome for testing RNG.
CONG
The linear congruential generator: LCG (16807, 0, 231 − 1)

2 SHR
2 shift register generators : F (250, 103, ⊕), F (1279, 1063, ⊕)

SWC
A subtract with carry generator: F (1279, 1063, −)

Roberto Innocente Random Numbers for Simulations

Ferrenberg: 2D Ising model montecarlo failure : CONG, 2
SHR, SWC, SWCW II

SWCW
A combined subtract with carry and Weyl generator

Roberto Innocente Random Numbers for Simulations

Closing consideration

RNGs are a swamp in which is easy to become trapped.

To devise a RNG :
Choose an algorithm already studied and for which theory can
predict the period
Consult publication about seed and parameters for it to get a
good rng
Test it thouroughly with TestU01 and DieHarder.
The modern view about the algorithms is that today probably an
LCG is not enough for current simulations, but can be conviniently
combined with one of the other algorithms like SHR or LFSR.
The Mersenne twister is now considered by many the best rng
around.
It can be it will be overtaken by WELL that was created by lEcuyer
and the japanese team of MT to solve some issues.

Roberto Innocente Random Numbers for Simulations

Lex aurea of RNG users

Never be first, never be last to use a RNG !

Paraphrased from pharmacology :
Never the first, never the last to prescribe a drug to patients !

Roberto Innocente Random Numbers for Simulations

Famous Quotes: Von Neumann, Knuth, Marsaglia, . . .

Von Neumann : Any one who considers arithmetical methods

of producing random digits is, of course, in a state of sin.
D.Knuth Random numbers should not be generated with a
method chosen at random.
Marsaglia : Random numbers are like sex: Even if they are
not very good, they are still pretty good.
Robert Coveyou The generation of random numbers is too
important to be left to chance.

Roberto Innocente Random Numbers for Simulations

quasi-Monte Carlo

A side argument :
quasi-random numbers (or sub-random)
quasi-Monte Carlo
Uses instead of random numbers low discrepancy numbers (or
quasi-random). Can converge faster if function is smooth.

Roberto Innocente Random Numbers for Simulations

Exercise buffon

1 Pick a programming language and OS of your choice

2 Choose an available rng
3 Write a program to simulate the Buffon experiment (if you are
able show it in graphic mode)
4 Report on a small table the sample size and the results

Roberto Innocente Random Numbers for Simulations

Exercises

Transform a uniform variate into one uniform over [a, b]

Transform a uniform variate into an Exponential variate PDF
λe −λx , CDF 1 − e −λx
Using Box-Muller transform a uniform variate in a normal one

Roberto Innocente Random Numbers for Simulations

Exercise Knuth's Super-random

1 Pick a programming language and OS of your choice

2 Implement Knuth's Super-random
3 Report on the flaws you find generating rn

Roberto Innocente Random Numbers for Simulations

Exercise NR Ran

Ran.h
Implement Numerical Recipes Ran and test it with dieharder ,

Roberto Innocente Random Numbers for Simulations

Exercise on MRG

Devise an MRG generator of maximum period and test it with

dieharder . Report it.

Roberto Innocente Random Numbers for Simulations

Exercise sprng

1 Choose a OS and a language between C++ and Fortran

2 Install SPRNG as detailed in previous slides
3 Make a program to check the simple interface both in serial
and in parallel mode
4 Make a program to check the full interface of SPRNG

Roberto Innocente Random Numbers for Simulations

Exercise kiss99

1 Pick a programming language and OS of your choice

2 Implement Marsaglia’s KISS99 rng
3 Report on the rn it generates
4 Test its randomness with DieHarder or TestU01

Roberto Innocente Random Numbers for Simulations

MLCG, LCG exercises

We know from theory how to produce a maximal period generator

in these 2 cases, but we should make an heuristic/ experimental
search for the good ones.
implement Lehmer generator and run dieharder on its output
produce an MLCG of maximal period m − 1
produce an LCG of maximal period m
check both with dieharder , eventually change them

Roberto Innocente Random Numbers for Simulations

Docker container for the exercises

A docker container based on ubuntu with everything you need

for the exercises and most of the algorithms and tests cited can be
run with :
docker run -u rng -it rinnocente/rng /bin/bash

Roberto Innocente Random Numbers for Simulations

References I

[1] Donald Knuth.

The Art of Computer Programming : vol 2. Seminumerical
Algorithms
2nd ed, 1981.
[2] Donald Knuth.
The Art of Computer Programming : vol 2. Seminumerical
Algorithms
4th ed, 1999.
[3] Press, Teukolsky, Vetterling, Flannery
Numerical Recipes - The Art of Scientific Computing
Cambridge University Press, 3e, 2007
[4] Gentle,J.E.
Random Number Generation and Monte Carlo methods
Springer, 2e, 2003

Roberto Innocente Random Numbers for Simulations

References II

[5] A.Rukhin, J. Soto, et al.

A Statistical Test Suite for Random and Pseudorandom Number
Generators for Cryptographic Applications
National Institute of Standards and Technology (D.Of
Commerce)
April 2010.
[6] H.Niederreiter, A.Winterhof
Applied Number Theory
Springer, 2015
[7] Crandall R., Pomerance C.
Prime Numbers - A computational Perspective
Springer, 2015

Roberto Innocente Random Numbers for Simulations

References III

[8] Solomon Wolf Golomb

Shift register sequences
Holden Day, 1967
[9] Solomon Wolf Golomb, Lloyd R. Welch, Richard M. Goldstein,
Alfred W. Hales
Shift Register Sequences
Aegean Park Press, 1982
[10] Menezes, van Oorschot, Vanstone
CRC Handbook of Applied Cryptography
CRC Press, 1996

Roberto Innocente Random Numbers for Simulations

References IV

[11] Lehmer, D. H.
Mathematical methods in large-scale computing units
Proceedings of a Second Symposium on Large-Scale Digital
Calculating Machinery: 141146. MR 0044899. 1949.
journal : Annals of the Computation Laboratory of Harvard
University, Vol. 26 (1951)).
[12] G. Marsaglia.
Random numbers fall mainly in the planes.
PNAS, 61 (1): 2528. 1968.
[13] A new class of Random Number generators (AWC,SWB)
G.Marsaglia, Zaman
The Annals of Applied Probability, 1991, n. 3

Roberto Innocente Random Numbers for Simulations

References V

[14] Tables of Linear Conguential Generators of different sizes and

good lattice structure
P. L'Ecuyer
Mathematics of Computation
Volume 68, Number 225, Januiary 1999
[15] Pierre L'Ecuyer , Richard Simard
TestU01: A Software Library in ANSI C for Empirical Testing of
Random Number Generators
ACM Transactions on Mathematical Software
33: 22. 2007.
[16] Pierre L'Ecuyer
Uniform random number generation.
Handbooks in Operations Research and Management Science
13:55-81, 2006

Roberto Innocente Random Numbers for Simulations

References VI

[17] Matsumoto, M.; Nishimura, T.

Mersenne twister: a 623-dimensionally equidistributed uniform
pseudo-random number generator
ACM Transactions on Modeling and Computer Simulation
8 (1): 330. doi:10.1145/272991.272995. 1998.
[18] Mutsuo Saito, Makoto Matsumoto
A variant of mersenne twister suitable for graphic processors.
CoRR, abs/1005.4972, 2010
[19] Robert G. Brown
Dieharder: A random number test suite
http://www.phy.duke.edu/˜rgb/General/dieharder.php , 2009

Roberto Innocente Random Numbers for Simulations

References VII

[20] Alan M.Ferrenberg, D.P.Landau and Y.Joanna Wong

Monte Carlo simulations: Hidden errors from ”good” random
number generators
Phys.Rev.Lett. 69,3382(1992)
[21] Panneton, F. O., l'Ecuyer P., Matsumoto M.
Improved long-period generators based on linear recurrences
modulo 2 (WELL)
ACM Transactions on Mathematical Software. 32 (1): 116.
doi:10.1145/1132973.1132974
[22] G. E. P. Box and Mervin E. Muller
A Note on the Generation of Random Normal Deviates
The Annals of Mathematical Statistics (1958)
Vol. 29, No. 2 pp. 610611

Roberto Innocente Random Numbers for Simulations

References VIII

[23] S.Park, K.Miller

Random numbers generators : good ones are hard to find
Communications of the ACM CACM Volume 31 Issue 10, Oct.
1988 Pages 1192-1201
[24] Richard Brent
Search for primitive trinomials (mod 2)
http://maths-people.anu.edu.au/~brent/trinom.html ,
2008
[25] Richard Brent, Paul Zimmermann
TWELVE NEW PRIMITIVE BINARY TRINOMIALS
https://arxiv.org/pdf/1605.09213.pdf

Roberto Innocente Random Numbers for Simulations

References IX

[26] Lewis, Payne

Generalized Feedback Shift Register Pseudorandom Number
Algorithm
Journal of the ACM Volume 20 Issue 3, July 1973 Pages 456-468
[27] Matsumoto, Kurita
Twisted GFSR Generators
http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/
ARTICLES/tgfsr3.pdf
April 1992

Roberto Innocente Random Numbers for Simulations

END

E N D

Roberto Innocente Random Numbers for Simulations