Part6_AnalyzingParallelPerformance
Part6_AnalyzingParallelPerformance
Objectives
At the end of this module, you should be able to
Define speedup and efficiency
Use Amdahl’s Law to predict maximum speedup
Use the Karp-Flatt metric to
analyze parallel program performance
predict speedup with additional processors
Speedup
Speedup is the ratio between sequential execution time and
parallel execution time
For example, if the sequential program executes in 6
seconds and the parallel program executes in 2 seconds, the
speedup is 3
y=x
Speedup
Speedup curves
look like this
Processors
Efficiency
Efficiency
A measure of processor utilization
Speedup divided by the number of processors
Example
Program achieves speedup of 3 on 4 CPUs
Efficiency is 3 / 4 = 75%
y = 1.0
Efficiency
Efficiency curves
look like this
Processors
Analyzing Parallel Performance
4
Portion of computation
that will be performed
sequentially
Execution Time
f Portion of computation
that will be executed
in parallel
f
1-f f
f f
(1-f )/2
(1-f )/3 (1-f )/4
(1-f )/5
Processors
Analyzing Parallel Performance
5
f (1 f ) 1
f (1 f ) / p f (1 f ) / p
Parallel overhead
Execution Time
increases with
# of processors
Processors
Analyzing Parallel Performance
8
Working time
Waiting time
Task completed
Analyzing Parallel Performance
9
Execution Time
Time lost
due to
workload
imbalance
Processors
Analyzing Parallel Performance
10
( n) ( n)
( n, p )
( n ) ( n ) / p ( n, p )
Analyzing Parallel Performance
11
( n) ( n)
( n, p )
( n ) ( n ) / p ( n, p )
Assumes parallel
work divides perfectly
among available CPUs
( n) ( n)
( n, p )
( n ) ( n ) / p ( n, p )
As n these
terms dominate
n = 100,000
Speedup
n = 10,000
n = 1,000
Processors
Analyzing Parallel Performance
14
3 1.8 60%
4 2 50%
( n) ( n)
( n, p )
( n ) ( n ) / p ( n, p )
The denominator represents parallel execution time
One processor does sequential code; others idle
All processors incur overhead time
Karp-Flatt Metric
1 / 1 / p
e
1 1/ p
The experimentally determined serial fraction is a
function of speedup and the number of processors
We can use e to determine whether efficiency
decreases are due to
Sequential component of computation
Increases in overhead
Case Study
When p = 2, e = 0.25
We know 0.15 of e is sequential component
Rest of e (0.05) is parallel overhead
If parallel overhead increases linearly with number of
processors, then it will be 0.15 when p = 3
We predict when p = 4, e = 0.30
Hence when p = 4, we predict speedup of 2.11
Superlinear Speedup
References