' $

Chapter 10

Simulation: An Introduction

& %
1
' $

Definition

• A simulation is an imitation of some real thing, state of

affairs, or process.
• The act of simulating something generally entails representing
certain key characteristics or behaviors of a selected physical or
abstract system. )

& %
2
' $

Use of Simulation
• Simulation is used in many contexts, including the modeling of
natural systems or human systems in order to gain insight into
their functioning.
• Simulation can be used to show the eventual real effects of
alternative conditions and courses of action.

& %
3
' $

Example 1: Checking distribution theory

Theory:
• A sample of size 4 is taken from a normal distribution with
mean 0.
• Consider the statistics
X̄
t= √
σ̂/ 4

• The statistic t follows a t-distribution with 3 degrees of

freedom.

& %
4
' $

Example 1
Simulation
• Generate 1000 random samples each of size 4 from a normal
distribution.
• For each sample, compute the value of the statistic t.
• Construct a histogram for these 1000 realizations of the
statistic t.

& %
5
' $

Example 1
Result:
• The histogram matches well with the t(3) distribution.

& %
6
' $

Example 2: Comparing estimators

We have learnt several robust estimators of location
Question: Which estimator is better, the trimmed mean or the
Winsorized mean?
Secondary question:
• Under what conditions(in terms of the underlying
distributions) is the estimator better?
• How to measure the performance? We may use the Mean
Square Error (MSE=Bias2 +Variance)
• It may be intractable to compute the MSE for each of the
estimators under different underlying distributions,
• Simulation is an alternative solution
• How to design such a simulation study?
& %
7
' $

Example 3: Buffon’s needle experiment

• A needle of length l is thrown randomly onto a grid of parallel
lines with distance d(> l)

& %
8
' $

Example 3
• What is the probability that the needle intersects a line?
• Answer:(2l)/πd
• Can we get the answer through simulations?
• A website gives the visualization of the experiment
http://www.metablake.com/pi.swf

& %
9
' $

Example 3
The experiment
• Simulate the throwing of a needle into a grid of parallel lines,
say N times
• Count the number of times the needle intersects a line, say n
times
• Then n/N gives estimate of the probability that the needle
intersects a line

& %
10
' $

How to do the simulation

• Step 1: Generate the position and the inclination of the needle
– Generate a random number, x, from Uniform (0,d/2).
This number represents the location of the centre of the
needle.
– Generate a random number, θ, from Uniform (0, π).
This number represents the angle of between the needle and
the parallel lines.

& %
11
' $

How to do the simulation

• Step 2: Check if the needle cuts a line.
– The needle cuts a line if x < (l/2)sin(θ)
– Create a variable w = 1 if x < (l/2)sin(θ) and 0 otherwise.
• Step 3: Repeat Steps 1 and 2 n times. Count the number of
times that w = 1, let say N times. Then an estimate of the
probability of the needle intersects a line is given by n/N .

& %
12
' $

Example 3: R Code
> # Buffon’s needle
> # X∼U(0,d/2), t∼U(0,pi) where d is the distance between 2
parallel lines
> # A needle of length L cut one of the lines if x<L/2*sin(t)
> # Theoretical Probability=2*L/(pi*d)
>ns=50000 ; d=2
>L=1 ; d2=d/2
> #Theoretical answer
>2*L/(pi*d)
[1]0.3183099
>x=runif(ns,0,d2)
>t=runif(ns,0,pi)
>length(x[x<L/2*sin(t)])/ns
[1] 0.3186
& %
13
' $

Random number generator

Definition
• A sequence of pseudo-random number {Ui } is a deterministic
sequence of number in [0,1] having the same relevant statistical
properties as a sequence of random numbers.

& %
14
' $
Congruential generators
• Congruential generators are defined by

Xi = (aXi−1 + c)modM

for a multiplier a, shift c, and modulus M.

• a,c and M are all integers
• Uniform random numbers are obtained by Ui = Xi /M
• To initialize, we have to provide a seed, X0
• If c = 0, generators having the form

Xi = aXi−1 modM

are called multiplicative congruential generator.

• if c > 0, they are called linear congruential generator.
& %
15
' $

Remarks
• M + 1 values {X0 ,X1 ,...XM } cannot be distinct and at least
one value must occur twice, as Xi and Xi+k , say.
• Xi , Xi+1 , ...Xi+k−1 is repeated as Xi+k , Xi+k+1 , ...Xi+2k−1 .
• The sequence Xi is periodic with period k ≤ M .
• For multiplicative generators, the maximal period is M − 1.
• If 0 ever occurs, it is repeated indefinitely.
• One of our primary objectives is to use a generator with as
large period as possible.
• However, a large period does not guarantee a good generator.

& %
16
' $

R function for congruential generators

> #Linear congruential generators
>lcg=function(n,a,m,c,x0){
+ran=NULL
+for (i in 1:n){
+x1=(a*x0+c)%%m
+x0=x1
+ran=c(ran, x1/m)}
+ran}
> lcg(10,397204094,2∧31-1,0,1234)
[1] 0.24381116 0.08947511 0.38319371 0.72800325
0.72792771 0.17503827
[7] 0.40680994 0.90923700 0.34271140 0.55960111

& %
17
' $

Generate uniform random numbers: SAS

* Generate Uniform random numbers;
data try1;
seed=1234;
do i =1 to 10;
x=ranuni(seed);
output;
end;
keep x;
run;
proc print data=try1;
var x;
run;

& %
18
' $

Generate uniform random numbers: R

> # Generate 1000 random numbers from U(0,1) distribution
> x=runif(1000)

In general, “runif(n,a,b)” generates a vector of n random numbers

from a uniform distribution between a and b.

& %
19
' $

Generate non-uniform random numbers

Inversion method
• If X has a continuous distribution function F (x) (i.e.
Pr(X ≤ x)), then F (X) ∼Uniform (0,1)
Algorithm:
• Generate U from Uniform (0,1)
• Set X = F −1 (U ) provided the inverse exists.

& %
20
' $

Exponential distribution
• If X follows an exponential distribution with parameter λ
(i.e.E(X) = λ), then
∫ x ∫ x/λ
1 t
F (x) = P r(X ≤ x) = e λ dt = e−y dy = 1 − e−x/λ
0 λ 0

Solving u = F (x) = 1 − e−x/λ for x, we have

x = F −1 (u) = −λ ∗ log(1 − u). Then
• Generate U from Uniform (0,1)
• Set X = −λ ∗ log(1 − U ) or X = −λ ∗ log(U )

& %
21
' $

Weibull distribution
• If X follows a Weibull distribution with parameter β, then it
can be shown that

F (x) = 1 − exp(−xβ ) on (0, ∞)

Note:f (x) = βxβ−1 exp(−xβ ) for x in (0, ∞)

Solving u = F (x) = 1 − exp(−xβ ) for x, we have
x = (−log(1 − u))1/β . Then
• Generate U from Uniform (0,1)
• Set X = (−log(1 − U ))1/β or X = (−log(U ))1/β .

& %
22
' $

Cauchy distribution
• If X follows a Cauchy distribution with parameter µ and σ,
then it can be shown that
1 1 x−µ
F (x) = + tan−1 ( )
2 π σ
for x in (−∞, ∞) Note: f (x) = 1
πσ(1+( x−µ 2
.
σ ) )

Solving u = F (x) = 12 + π1 tan−1 ( x−µ

σ ), we have
x = σtan[π(u − 0.5)] + µ.
• Generate U from Uniform (0,1)
• Set X = σtan[π(U − 0.5)] + µ.

& %
23
' $

Algorithm to generate a normal random variable

Box-Muller Algorithm

• Generate U1 and U2 from Uniform (0,1)

• Set θ = 2πU1 and R = (−2logU2 )1/2
• Set X = R cos(θ) and Y = R sin(θ)

Then X and Y are independent standard normal variables. Often,

only X or Y is used.

& %
24
' $
Algorithm to generate a normal random variable
Polar algorithm (Modified Box-Muller algorithm)

• Generate U1 , U2 ∼ Uniform(−1, 1) until U12 + U22 < 1

√
• Set W = U1 + U2 and c = −2log(W )/W
2 2

• Set X = cU1 and Y = cU2

Then X and Y are independent standard normal variables.

Remark: Polar algorithm uses rejection to avoid calculating two

trigonometric functions and so is usually substantially faster
compared to Box-Muller algorithm. However, using (2n) uniform
random numbers will not generate (2n) standard normal random
numbers.
& %
25
' $

Generate a random variable from other r. v.

Cauchy distribution
• If Y and Z are independent and follow N(0,1), then X = Y /Z
follows a Cauchy(0,1) distribution
• If Y ∼ N (µ, σ 2 ) and Z ∼ N (0, 1), and are independent, then
X = Y /Z follows a Cauchy (µ, σ 2 ) distribution

& %
26
' $

Generate a random variable from other r. v.

Chi-square distribution
• If Y follows a normal distribution, then X = Y 2 follows a
Chi-square distribution with 1 degree of freedom
• If Y1 , Y2 , · · · , Yn are independent and identically distributed
standard normal variables, then

X = Y12 + Y22 + · · · + Yn2

follows χ2 (n), a Chi-square distribution with n degrees of

freedom.

& %
27
' $

Generate a random variable from other r. v.

Student’s t-distribution
• If Y ∼ N (0, 1) and Z ∼ χ2 (p), then
Y
X= √
Z/p
follows a Student’s t distribution with p degrees of freedom

& %
28
' $

Generate a random variable from other r. v.

F distribution
• If Y ∼ χ2 (m) and Z ∼ χ2 (n), then
Y /m
X=
Z/n
follows a F distribution with degrees of freedom m and n.

& %
29
' $

Function to generate uniform distribution r. v.

To generate random numbers from Uniform (a,b)
1
f (x) = for a < x < b
(b − a)
In R
> # Generate uniform r. v.
> n=100
> a=0
> b=100
> x=runif(n,a,b)
>x

& %
30
' $

Function to generate uniform distribution r. v.

In SAS
data unif;
seed=1234;
n=100;a=0;b=10;
do i=1 to n;
x=a+(b-a)* ranuni(seed);
output;
end;
keep x;
run;

& %
31
' $

Function to generate Normal distribution r. v.

To generate random numbers from Normal (µ, σ 2 ) for −∞ <x< ∞
1 (x − µ)2
f (x) = √ exp(− 2
) for − ∞ < x < ∞
2πσ 2σ
In R
> # Generate normal r. v.
> n=100
> mu=0
> sigma=1
> x=rnorm(n,mean=mu,sd=sigma)
>x

& %
32
' $

Function to generate Normal distribution r. v.

In SAS
data norm;
seed=1234;
n=100;mu=0;sigma=1;
do i=1 to n;
x=mu+sigma*rannor(seed);
output;
end;
keep x;
run;

& %
33
' $

Function to generate Expo distribution r. v.

To generate random numbers from Exponential (λ)
1 x
f (x) = exp(− ) for x > 0
λ λ
In R
> # Generate exponential r. v.
> n=100
> lambda=5
> x=rexp(n,rate=lambda)
>x

& %
34
' $

Function to generate Expo distribution r. v.

In SAS
data expno;
seed=1234;
n=100;lambda=5;
do i=1 to n;
x=lambda*ranexp(seed);
output;
end;
keep x;
run;

& %
35
' $

Function to generate Gamma distribution r. v.

To generate random numbers from Gamma (α, β)
1 α−1 x
f (x) = α x exp(− ) for x > 0
β Γ(α) β
In R
> # Generate gamma r. v.
> n=100
> alpha=1
> beta=2
> x=rgamma(n,shape=alpha,scale=beta)
>x

& %
36
' $

Function to generate Gamma distribution r. v.

In SAS
data gammano;
seed=1234;
n=100;alpha=1;beta=2;
do i=1 to n;
x=beta*rangam(seed,alpha);
output;
end;
keep x;
run;

& %
37
' $

Function to generate χ2 distribution r. v.

To generate random numbers from χ2 (p)
1 p x
f (x) = p p x 2 −1 exp(− ) for x > 0
2 2 Γ( 2 ) 2

In R
> # Generate Chi-square r. v.
> n=100
> p=10
> x=rchisq(n,df=p)
>x

& %
38
' $

Function to generate χ2 distribution r. v.

In SAS
data chisqno;
seed=1234;
n=100;df=10;alpha=df/2;
do i=1 to n;
x=2*rangam(seed,alpha);
output;
end;
keep x;
run;

& %
39
' $

Function to generate Beta distribution r. v.

To generate random numbers from Beta (α, β)
Γ(α + β) α−1
f (x) = x (1 − x)β−1 for 0 < x < 1
Γ(α)Γ(β)
In R
> # Generate Beta r. v.
> n=100
> a=2
> b=3
> x=rbeta(n,shape1=a,shape2=b)
>x

& %
40
' $

Function to generate Beta distribution r. v.

In SAS
data betano;
seed=1234;
n=100;alpha=2;beta=3;
do i=1 to n;
y1=rangam(seed,alpha);
y2=rangam(seed,beta);
x=y1/(y1+y2);
output;
end;
keep x;
run;

& %
41
' $

Function to generate t-distribution r. v.

To generate random numbers from t(k)

Γ( k+1
2 ) 1 1
f (x) = k
√ 2 k+1
for − ∞ < x < ∞
Γ( 2 ) x
kπ (1 + k ) 2

In R
> # Generate t r. v.
> n=100
> k=5
> x=rt(n,df=k)
>x

& %
42
' $

Function to generate t distribution r. v.

In SAS
data tno;
seed=1234;
n=100;df=5;alpha=df/2;
do i=1 to n;
y1=rannor(seed);
y2=rangam(seed,alpha);
x=y1/sqrt(y2/df);
output;
end;
keep x;
run;

& %
43
' $

Function to generate F-distribution r.v.

To generate random numbers from F(m,n)
n1
Γ( n1 +n 2
) n1 n1 −1
x 2
f (x) = n1
2
n2 ( )2 n1 +n2 for 0 < x < ∞
Γ( 2 )Γ( 2 ) n2 (1 + n1
n2 x)
2

In R
> # Generate F r. v.
> n=100
> n1=5
> n2=10
> x=rf(n,df1=n1,df2=n2)
>x

& %
44
' $

Function to generate F distribution r.v.

In SAS
data fno;
seed=1234;
n=100; df1=5; df2=10;
do i=1 to n;
y1=2*rangam(seed,df1/2);
y2=2*rangam(seed,df2/2);
x=(y1/df1)/(y2/df2);
output;
end;
keep x;
run;

& %
45
' $

Function to generate Binomial distribution r.v.

To generate random numbers from Binomial(n,p)
 
n
f (x) =   px (1 − p)n−x for x = 0, 1, 2, · · · , n
x

In R
> # Generate Binomial r.v.
> nn=100
> n=10
> p=0.3
> x=rbinom(100,size=n,prob=p)
>x

& %
46
' $

Function to generate Binomial distribution r.v.

In SAS
data binomno;
seed=1234;
ns=100;n=10;p=0.3;
do i=1 to ns;
x=ranbin(seed,n,p);
output;
end;
keep x;
run;

& %
47
' $

Function to generate Poisson distribution r.v.

To generate random numbers from poisson(λ)

e−λ λx
f (x) = for x = 0, 1, 2, · · ·
x!
In R
> # Generate Poisson r. v.
> n=100
> lambda=3
> x=rpois(100,lambda)
>x

& %
48
' $

Function to generate Poisson distribution r.v.

In SAS
data poisno;
seed=1234;
n=100;lambda=3;
do i=1 to n;
x=ranpoi(seed,lambda);
output;
end;
keep x;
run;

& %
49
' $
Function to generate Hypergeometric r.v.
To generate random numbers from Hypergeometric distribution
(n, N, S)
  
S N −S
  
x n−x
f (x) =   for x = 0, 1, 2, · · · , min(n, S)
N
 
n

In R
> # Generate hypergeometric r.v.
> ns=100;n=10
> S=20;N=50
> x=rhyper(ns,S,N,n)
>x
& %
50
' $

Function to generate Nega-Binomial distr. r.v.

To generate random numbers from NBinom(r, p)
 
r+x−1
f (x) =   pr (1 − p)x for x = 0, 1, 2, · · ·
x

In R
> # Generate Negative Binomial r. v.
> n=100
> r=10
> p=0.3
> x=rnbinom(n,size=r,prob=p)
>x

& %
51