Introduction to Parallel Programming – Part 6

Analyzing Parallel Performance

Intel Software College
 Intel® Software College

Objectives
At the end of this module, you should be able to
Define speedup and efficiency
Use Amdahl’s Law to predict maximum speedup
Use the Karp-Flatt metric to
analyze parallel program performance
predict speedup with additional processors

Analyzing Parallel Performance

2

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Speedup
Speedup is the ratio between sequential execution time and
parallel execution time
For example, if the sequential program executes in 6
seconds and the parallel program executes in 2 seconds, the
speedup is 3

y=x
Speedup

Speedup curves
look like this

Processors

Analyzing Parallel Performance

3

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Efficiency

Efficiency
A measure of processor utilization
Speedup divided by the number of processors
Example
Program achieves speedup of 3 on 4 CPUs
Efficiency is 3 / 4 = 75%

y = 1.0
Efficiency

Efficiency curves
look like this

Processors
Analyzing Parallel Performance
4

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Idea Behind Amdahl’s Law

Portion of computation
that will be performed
sequentially
Execution Time

f Portion of computation
that will be executed
in parallel
f
1-f f
f f
(1-f )/2
(1-f )/3 (1-f )/4
(1-f )/5

Processors
Analyzing Parallel Performance
5

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Derivation of Amdahl’s Law

Speedup is ratio of execution time on 1 processor to

execution time on p processors
Execution time on 1 processor is f + (1-f)
Execution time on p processors is at least f + (1-f)/p

f  (1  f ) 1
 
f  (1  f ) / p f  (1  f ) / p

Analyzing Parallel Performance

6

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Amdahl’s Law Is Too Optimistic

Amdahl’s Law ignores parallel processing overhead

Examples of this overhead include time spent
creating and terminating threads
Parallel processing overhead is usually an increasing
function of the number of processors

Analyzing Parallel Performance

7

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Graph with Parallel Overhead Added

Parallel overhead
Execution Time

increases with
# of processors

Processors
Analyzing Parallel Performance
8

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Other Optimistic Assumptions

Amdahl’s Law assumes that the computation divides

evenly among the processors
In reality, the amount of work does not divide evenly
among the processors
Processor waiting time is another form of overhead
Task started

Working time

Waiting time
Task completed
Analyzing Parallel Performance
9

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Graph with Workload Imbalance Added

Execution Time

Time lost
due to
workload
imbalance
Processors
Analyzing Parallel Performance
10

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

More General Speedup Formula

(n,p) Speedup for problem of size n on p CPUs

(n) Time spent in sequential portion of code
for problem of size n
(n) Time spent in parallelizable portion of
code for problem of size n
(n,p)Parallel overhead

 ( n)   ( n)
 ( n, p ) 
 ( n )   ( n ) / p   ( n, p )
Analyzing Parallel Performance
11

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Amdahl’s Law: Maximum Speedup

 ( n)   ( n)
 ( n, p ) 
 ( n )   ( n ) / p   ( n, p )

Assumes parallel
work divides perfectly
among available CPUs

This term is set to 0

Analyzing Parallel Performance

12

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

The Amdahl Effect

 ( n)   ( n)
 ( n, p ) 
 ( n )   ( n ) / p   ( n, p )

As n   these
terms dominate

Speedup is an increasing function of problem size

Analyzing Parallel Performance

13

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Illustration of the Amdahl Effect

Linear speedup

n = 100,000
Speedup

n = 10,000

n = 1,000

Processors
Analyzing Parallel Performance
14

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Using Amdahl’s Law

Program executes in 5 seconds

Profile reveals 80% of time spent in function alpha,
which we can execute in parallel
What would be maximum speedup on 2 processors?
1 1
  1.67
0.2  (1  0.2) / 2 0.6

New execution time ≥ 5 sec / 1.67 = 3 seconds

Analyzing Parallel Performance

15

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

The Karp-Flatt Metric

Suppose we benchmark a parallel program and get

these speedup figures

Processors Speedup Efficiency

2 1.5 75%

3 1.8 60%

4 2 50%

Why is efficiency dropping?

How much speedup could we expect on 8 processors?

Analyzing Parallel Performance

16

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Deriving the Karp-Flatt Metric

 ( n)   ( n)
 ( n, p ) 
 ( n )   ( n ) / p   ( n, p )
The denominator represents parallel execution time
One processor does sequential code; others idle
All processors incur overhead time

“Wasted time” = (p-1)(n) + p(n, p)

Experimentally determined serial fraction = “wasted
time” divided by (p-1) times sequential time
Analyzing Parallel Performance
17

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Karp-Flatt Metric

1 /  1 / p
e
1  1/ p
The experimentally determined serial fraction is a
function of speedup and the number of processors
We can use e to determine whether efficiency
decreases are due to
Sequential component of computation
Increases in overhead

Analyzing Parallel Performance

18

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

How to Interpret “e”

If “e” is constant as the number of processors

increases, then speedup is constrained by the
sequential component of the computation
If “e” is increasing as the number of processors
increases, then speedup is constrained by
parallel overhead, such as
Thread creation/termination time
Contention for shared data structures
Cache-related inefficiencies
Often a combination of the two factors

Analyzing Parallel Performance

19

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Going Back to Our Example

Processors Speedup Efficiency e

2 1.5 75% 0.33
3 1.8 60% 0.33
4 2.0 50% 0.33

In this case, speedup is constrained by the relatively

large amount of time spent in sequential code

Analyzing Parallel Performance

20

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Example: Rectangle Rule Program

Processors Speedup Efficiency e

2 1.87 93% 0.070

3 2.60 87% 0.078

4 3.16 79% 0.089

Benchmark data from an OpenMP program computing  using

the rectangle rule
We can predict speedup on 6 processors
Extrapolate e to be 0.11
Speedup would be 3.87

Analyzing Parallel Performance

21

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Speedup Prediction Formula

1 /  1 / p
e
1  1/ p
p
 
e( p  1)  1

Analyzing Parallel Performance

22

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Case Study

We benchmark a sequential program and find it

spends 85% of its time in functions we believe we
can make parallel
We make these functions multithreaded and execute
the program on a dual-core system
The parallel program achieves a speedup of 1.67 on
2 processors
If we can get access to a quad-core system, what
kind of speedup should we expect?

Analyzing Parallel Performance

23

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Prediction Based on Amdahl’s Law

1

0.15  (1  0.15) / 4
  2.76

Analyzing Parallel Performance

24

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Prediction Based on Karp-Flatt Metric

When p = 2, e = 0.25
We know 0.15 of e is sequential component
Rest of e (0.05) is parallel overhead
If parallel overhead increases linearly with number of
processors, then it will be 0.15 when p = 3
We predict when p = 4, e = 0.30
Hence when p = 4, we predict speedup of 2.11

Analyzing Parallel Performance

25

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Superlinear Speedup

According to our general speedup formula, the

maximum speedup a program can achieve on p
processors is p
Superlinear speedup is the situation where
speedup is greater than the number of processors
used
It means the computational rate of the processors is
faster when the parallel program is executing
Superlinear speedup is usually caused because the
cache hit rate of the parallel program is higher

Analyzing Parallel Performance

26

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

References

Michael J. Quinn, Parallel Programming in C with MPI

and OpenMP, McGraw-Hill (2004).

Analyzing Parallel Performance

27

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
 Intel® Software College

Analyzing Parallel Performance

28

Copyright © 2006, Intel Corporation. All rights reserved.

Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.