CPSC 531:

System Modeling and Simulation

Carey Williamson
Department of Computer Science
University of Calgary
Fall 2017
 Agenda

 Welcome!
 Course Overview
 Learning Outcomes
 Administrative Details
 Expectations
 Q&A

 Today’s Lecture Material:

— Basics of Modeling
— Computer Systems Performance Evaluation
— Simulation Modeling Example
2
 Course Overview

 The purpose of this course is to learn the basics of

systems modeling, discrete-event simulation, and
computer systems performance evaluation.
 Syllabus:
— Intro to modeling and simulation (1 week)
— Random numbers and Monte Carlo simulation (1 week)
— Probability models: discrete and continuous (2 weeks)
— Discrete-event simulation methods (2 weeks)
— Simulation output analysis (2 weeks)
— Simulation input analysis (2 weeks)
— Queueing theory and Markov chains (2 weeks)

3
 Learning Outcomes

 Classify/describe different types of simulations.

 Solve simple problems using Monte Carlo methods.
 Develop discrete-event simulation models.
 Generate random variates using inverse transforms.
 Understand differences between transient and
steady-state behaviour of a system.
 Compute confidence intervals from simulation data.
 Conduct goodness of fit tests for input data models.
 Use basic queueing theory for simple system models.

4
 Administrative Details

 Course Web Site:

http://www.cpsc.ucalgary.ca/~carey/CPSC531
 D2L Site: site exists, with actual content coming soon!
 Lectures: Tue/Thur 9:30am-10:45am MS 211
 Instructor: Carey Williamson
 Tutorials: Tue/Thur 1:00pm-1:50pm SS 006
 TA: Jonathan Hudson (tutorials start week of Sept 18)
 Textbook: Discrete-Event Simulation: A First Course
 Grading:
— Assignments (40%) - four programming assignments
— Midterm Exam (20%) - in-class on Thursday, October 26
— Final Exam (40%) - two-hour closed book exam
 Grading Scheme: threshold-based step-function 5
 Instructor

 Professor: Carey Williamson

 Department: Computer Science
 Office: ICT 736
 Office Hours: Mondays (1:00pm-3:00pm) or by appt
 Phone: (403) 220-6780
 Email: [email protected] (any time!)
 Web Site: http://www.cpsc.ucalgary.ca/~carey
 Research Interests: Computer networks, Computer
systems performance evaluation, Network simulation
 Hobbies: Golf, curling, hiking, travel, cooking, family
6
 Agenda

 Welcome!
 Course Overview
 Learning Outcomes
 Administrative Details
 Expectations
 Q&A

 Today’s Lecture Material:

— Basics of Modeling
— Computer Systems Performance Evaluation
— Simulation Modeling Example
7
 Basics of Modeling

 What is a model?
— An abstract representation of a (real) system that captures
the essential characteristics or properties of the system
— Often requires making simplifying assumptions about how
the system actually works
 Examples:
— Model airplane; molecular model; performance model

 Modeling is an essential tool in computer system

performance evaluation (as we will see shortly)
 Note that modeling is both an ‘art’ and a ‘science’
8
 Modeling: A Reality Check

 A famous quote:

“All models are wrong; some models are useful.”

- George Box, 1976

 Models are useful when they provide critical insights

into the system behaviour (e.g., its performance)
 Models are especially valuable when they are simple,
elegant, and computationally fast

9
 Computer Systems Performance Evaluation

 Performance is a key consideration in the design,

procurement, and use of computer systems.
 The typical goal is to get the highest possible
performance for a given cost (e.g., dollars, energy)

 Performance evaluation is a well-defined sub-domain

of computer science that has been around for 40 yrs
 Need basic knowledge of the tools and techniques of
computer systems performance evaluation
— What are the performance requirements?
— How to compare different system alternatives?
10
 Objectives of Performance Evaluation

Establish a quantitative understanding of system behaviour

This understanding should be sufficient for:

 Evaluating alternative system designs/configurations
— e.g., should our Web site run on one server or two servers?
— e.g., should Web server software be Apache, IIS, or nginx?
 Predicting system performance for a given set of inputs
— e.g., predict the mean response time of a Web server when the
number of users is increased
 Performance debugging and system tuning
— e.g., identify/remove bottlenecks, optimize configuration
— e.g., why is D2L so slow? is it the server, or the network?

11
 Approaches to Performance Evaluation

Three main approaches:

1. Experimental
— Obtain measurement data by observing the events and activities
on an existing system; evaluate new algorithms or designs by
implementing and comparing them in a real system
2. Simulation modeling
— Develop a computer program that implements an abstracted
model of the physical system; manipulate the model and/or its
inputs to estimate the system performance (e.g., randomization)
3. Analytical modeling
— Represent the system by an abstract mathematical model of the
physical system (e.g., formula); manipulate parameters of the
model to obtain information about system performance
12
 High Level Overview

Performance Evaluation

Performance Performance Modeling

Measurement

Analytic Modeling Simulation

13
 Performance Measurement

 Measure the performance directly on a system

 Need to characterize the workload placed on the
system during measurement
 Generally provides the most valid results
 Nevertheless, not very flexible
— May be difficult (or even impossible) to vary some
workload parameters

14
 Performance Modeling

 Construct a model
— An abstracted representation of a system obtained by
making assumptions about how the system works
— Captures the most salient characteristics of the system
 Reasons for using models
— Experimenting with the real system may be
 too costly
 too risky, or
 too disruptive to system operation
— System may not even exist yet (e.g., planning stage)

15
 Analytic Modeling

 Mathematical methods are used to obtain solutions

to the performance measures of interest
 Examples: queueing models for computer systems or
computer communication networks
 Numerical results are easy to compute if a simple
analytic solution is available
 Useful approach when one only needs rough
estimates of performance measures
 Solutions to complex models may be difficult to
obtain

16
 Simulation Modeling

 Develop a simulation program that implements the

model
 Run the simulation program and use the data
collected to estimate the performance measures of
interest (typically using randomization)
 A system can be studied at an arbitrary level of detail
 It may be costly to develop and run the simulation
program

17
 Advantages of Simulation

 New policies and procedures can be explored without

disrupting the ongoing operation of the real system
 New designs can be tested without committing
resources for their acquisition
 Time can be compressed or expanded to allow for a
speed-up or slow-down of the phenomenon under
study
 Insight can be obtained about the interactions of
variables, and which ones have the most impact on
system performance
 Can obtain answers to “What if…” questions
18
 Areas of Application for Simulation

 Manufacturing applications
 Financial markets
 Military applications
 Logistics and supply chain management
 Transportation modes and traffic
 Business process simulation
 Health care optimization
 Facility placement problems
 Communication networks
 And many more!
19
 Simulation Example

 Hair salon (e.g., Witchcraft)

 Two stylists (one fast, one slow)
 Limited size waiting room (N chairs)
 Customers arrive at random times (no appts)
 Customers are impatient (don’t like waiting long)

 Question: How many customers do you lose by not

having enough chairs in the waiting room?

20
 Agenda

 Welcome!
 Course Overview
 Learning Outcomes
 Administrative Details
 Expectations
 Q&A

 Today’s Lecture Material:

— Basics of Modeling
— Computer Systems Performance Evaluation
— Simulation Modeling Example
21