Introduction to Software

Testing
(2nd edition)
Chapter 7.1, 7.2

Overview Graph Coverage

Criteria
Paul Ammann & Jeff Offutt

http://www.cs.gmu.edu/~offutt/softw
aretest/

Update, January 2016

 Ch. 7 : Graph Coverage
Four Structures for
Modeling Software

Input Graph Logic Synta

Space Applie s Applied x
d to to
Applie
Sourc FSMs
d to
e
Specs DNF

Source Specs Sourc Model

e s
Design Use Integ Input
Introduction to Software Testing, Edition 2 (Ch 07) cases © Ammann & Offutt 2
 Covering Graphs (7.1)
• Graphs are the most commonly used structure
for testing

• Graphs can come from many sources

– Control flow graphs
– Design structure
– FSMs and statecharts
– Use cases

• Tests usually are intended to “cover” the graph

in some way

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 3

 Definition of a Graph
• A set N of nodes, N is not empty

• A set N of initial nodes, N is not empty

0 0

• A set N of final nodes, N is not empty

f f

• A set E of edges, each edge from one node to

another
– ( ni , nj ), i is predecessor, j is successor

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 4

 Example Graphs
1 1 2 3 1

Not a
valid
graph
2 3 4 5 6 7 2 3

4 8 9 10 4

N0 = { 1} N0 = { 1, 2, 3 } N0 = { }
Nf = { 4 } Nf = { 8, 9, 10 } Nf = { 4 }
E = { (1,2), E = { (1,4), (1,5), (2,5), (3,6), E = { (1,2),
(1,3), (2,4), (3, 7), (4, 8), (5,8), (5,9), (1,3), (2,4),
Introduction(3,4) } Edition 2 (Ch 07) (6,2), (6,10), (7,10)
to Software Testing, (9,6)
© Ammann & Offutt } (3,4) } 5
 Paths in Graphs
• Path : A sequence of nodes – [n , n , …, n ]
1 2 M
– Each pair of nodes is an edge
• Length : The number of edges
– A single node is a path of length 0
• Subpath : A subsequence of nodes in p is a
subpath of p

1 2 3
A Few Paths
[ 1, 4, 8 ]
4 5 6 7 [ 2, 5, 9, 6, 2 ]
[ 3, 7, 10 ]

8 9 10
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 6
 Test Paths and SESEs
• Test Path : A path that starts at an initial node
and ends at a final node
• Test paths represent execution of test cases
– Some test paths can be executed by many tests
– Some test paths cannot be executed by any tests
• SESE graphs : All test paths start at a single
node and end at another node
– Single-entry, single-exit
– N0 and Nf have exactly one node
Double-diamond
2 5 graph
Four test paths
1 4 7
[1, 2, 4, 5, 7]
3 6 [1, 2, 4, 6, 7]
[1, 3, 4, 5, 7]
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt
[1, 3, 4, 6, 7] 7
 Visiting and Touring
• Visit : A test path p visits node n if n is in p
A test path p visits edge e if e is in p
• Tour : A test path p tours subpath q if q is a
subpath of p
Path [ 1, 2, 4, 5, 7 ]
Visits nodes 1, 2, 4, 5, 7
Visits edges (1, 2), (2, 4), (4, 5), (5, 7)
Tours subpaths [1, 2, 4], [2, 4, 5], [4, 5,
7], [1, 2, 4, 5], [2, 4, 5, 7], [1, 2, 4, 5, 7]
(Also, each edge is technically a subpath)

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 8

 Tests and Test Paths
• path (t) : The test path executed by test t
• path (T) : The set of test paths executed by
the set of tests T
• Each test executes one and only one test path
– Complete execution from a start node to an final
node
• A location in a graph (node or edge) can be
reached from another location if there is a
sequence of edges from the first location to
the second
– Syntactic reach : A subpath exists in the graph
– Semantic reach : A test exists that can execute that
subpath
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 9
 Tests and Test Paths
test many-to-
1 one
Test
test
Path
2
test
3
Deterministic software–test always executes the
same test path
many-to-many
test Test Path
1 1
test Test Path
2 2
test Test Path
3 3
Non-deterministic software–the same test can
execute different test paths
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 10
Testing and Covering Graphs (7.2)
• We use graphs in testing as follows :
– Develop a model of the software as a graph
– Require tests to visit or tour specific sets of nodes,
edges or subpaths
• Test Requirements (TR) : Describe properties of test
paths
• Test Criterion : Rules that define test requirements
• Satisfaction : Given a set TR of test requirements for a
criterion C, a set of tests T satisfies C on a graph if
• and only ifCoverage
Structural for every Criteria
test requirement in TR,
: Defined on therejust
a graph is a
test pathof
in terms innodes
path(T) that
and meets the test requirement tr
edges
• Data Flow Coverage Criteria : Requires a graph to be
annotated with references to variables

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 11

 Node and Edge Coverage
• The first (and simplest) two criteria require
that each node and edge in a graph be
executed
Node Coverage (NC) : Test set T satisfies node
coverage on graph G iff for every syntactically
reachable node n in N, there is some path p in
path(T) such that p visits n.
• This statement is a bit cumbersome, so we abbreviate
it in terms of the set of test requirements

Node Coverage (NC) : TR contains each

reachable node in G.
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 12
 Node and Edge Coverage
• Edge coverage is slightly stronger than node
coverage
Edge Coverage (EC) : TR contains each
reachable path of length up to 1, inclusive, in
G.
• The phrase “length up to 1” allows for graphs
with one node and no edges
• NC and EC are only different when there is an
edge and another subpath between a pair of
nodes (as in an “if-else” statement)
Node Coverage : TR = { 1, 2, 3 }
1 Test Path = [ 1,
2, 3 ]
2
Edge Coverage : TR = { (1, 2), (1,
3
3), (2, 3) }
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt Test Paths = [ 1, 13
 Paths of Length 1 and 0
• A graph with only one node will not have any
edges
1

• It may seem trivial, but formally, Edge

Coverage needs to require Node Coverage on
• this graph Edge Coverage will not subsume
Otherwise,
Node Coverage
– So we define “length up to 1” instead of simply
“length 1”
• We have the same issue with
1
graphs that only have one edge
– for Edge-Pair Coverage …
2

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 14

 Covering Multiple Edges
• Edge-pair coverage requires pairs of edges, or
subpaths of length 2
Edge-Pair Coverage (EPC) : TR contains each
reachable path of length up to 2, inclusive, in
G.
• The phrase “length up to 2” is used to include
graphs that have less than 2 edges
1 5
Edge-Pair Coverage :
2 4 TR = { [1,4,5], [1,4,6],
[2,4,5], [2,4,6], [3,4,5],
[3,4,6] }
3 6

• The logical extension is to require all paths …

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 15

 Covering Multiple Edges

Complete Path Coverage (CPC) : TR contains all

paths in G.

Unfortunately, this is impossible if the graph has

a loop, so a weak compromise makes the tester
decide which paths:

Specified Path Coverage (SPC) : TR contains a

set S of test paths, where S is supplied as a
parameter.

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 16

 Structural Coverage Example
Node Coverage
TR = { 1, 2, 3, 4, 5, 6, 7 }
Test Paths: [ 1, 2, 3, 4, 7 ] [ 1, 2, 3, 5, 6, 5,
1 7]
Edge Coverage
TR = { (1,2), (1, 3), (2, 3), (3, 4), (3, 5), (4,
2 7), (5, 6), (5, 7), (6, 5) }
Test Paths: [ 1, 2, 3, 4, 7 ] [1, 3, 5, 6, 5, 7 ]
3
Edge-Pair Coverage
TR = {[1,2,3], [1,3,4], [1,3,5], [2,3,4],
4 5 [2,3,5], [3,4,7],
[3,5,6], [3,5,7], [5,6,5], [6,5,6],
[6,5,7] }
6
7 Test Paths: [ 1, 2, 3, 4, 7 ] [ 1, 2, 3, 5, 7 ]
[ 1, 3, 4, 7Complete
] Path Coverage
Test Paths: [ [1,1,2,3,3,5,4,6,7 5,
] [6,
1,5,
2,73,
] 5, 7 ]
[ 1, 2, 3, 5, 6, 5, 6 ] [ 1, 2, 3, 5, 6, 5, 6, 5, 7 ]
[ 1, 2, 3, 5, 6, 5, 6, 5, 6, 5, 7 ] …
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 17
 Handling Loops in Graphs
• If a graph contains a loop, it has an infinite
number of paths

• Thus, CPC is not feasible

• SPC is not satisfactory because the results are

subjective and vary with the tester

• Attempts to “deal with” loops:

– 1970s : Execute cycles once ([4, 5, 4] in previous example,
informal)
– 1980s : Execute each loop, exactly once (formalized)
– 1990s : Execute loops 0 times, once, more than once (informal
description)
– 2000s : Prime paths (touring, sidetrips, and detours)
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 18
 Simple Paths and Prime Paths
• Simple Path : A path from node ni to nj is
simple if no node appears more than once,
except possibly the first and last nodes are the
same
– No internal loops
– A loop is a simple path
• Prime Path : A simple path that does not
appear as a proper subpath of any other
simple path Simple Paths : [1,2,4,1], [1,3,4,1],
[2,4,1,2], [2,4,1,3], [3,4,1,2], [3,4,1,3],
1
[4,1,2,4], [4,1,3,4], [1,2,4], [1,3,4],
2 3 [2,4,1], [3,4,1], [4,1,2], [4,1,3], [1,2],
4 [1,3], [2,4], [3,4], [4,1], [1], [2], [3], [4]

Prime Paths : [2,4,1,2], [2,4,1,3],

[1,3,4,1], [1,2,4,1], [3,4,1,2], [4,1,3,4],
Introduction to Software Testing, Edition 2 (Ch 07)
[4,1,2,4], [3,4,1,3]
© Ammann & Offutt 19
 Prime Path Coverage
• A simple, elegant and finite criterion that
requires loops to be executed as well as
skipped
Prime Path Coverage (PPC) : TR contains each
prime path in G.

• Will tour all paths of length 0, 1, …

• That is, it subsumes node and edge coverage
• PPC almost, but not quite, subsumes EPC …

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 20

 PPC Does Not Subsume EPC
• If a node n has an edge to itself (self
edge), EPC requires [n, n, m] and [m, n,
n]
• [n, n, m] is not prime
• Neither [n, n, m] nor [m, n, n] are simple
paths (not prime)
EPC Requirements :
1 TR = { [1,2,3], [1,2,2], [2,2,3],
[2,2,2] }

2 PPC Requirements
:TR = { [1,2,3],

3 [2,2] }

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 21

 Prime Path Example
• The previous example has 38 simple paths
• Only nine prime paths

1
Prime Paths
2 [1, 2, 3, 4, 7]
[1, 2, 3, 5, 7] Execute
[1, 2, 3, 5, 6] loop 0 times
3 [1, 3, 4, 7]
[1, 3, 5, 7] Execute
4 5 [1, 3, 5, 6] loop once
[6, 5, 7]
6 [6, 5, 6] Execute loop
7 [5, 6, 5] more than once

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 22

 Touring, Sidetrips, and Detours
• Prime paths do not have internal loops … test
paths might
• Tour : A test path p tours subpath q if q is a
subpath of p
• Tour With Sidetrips : A test path p tours
subpath q with sidetrips iff every edge in q is
also in p in the same order
• The tour can include a sidetrip, as long as it comes
back to the same node
• Tour With Detours : A test path p tours
subpath q with detours iff every node in q is
also in p in the same order
• The tour can include a detour from node ni, as long
as it comes back to the prime path at a successor
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 23
 Sidetrips and Detours Example
1 2 3 4
1 2 3 5 6

Touring the prime

path [1, 2, 3, 5, 6] 4
without sidetrips or
detours 1 2 5 6
1 2 3 5 6
3 4
Touring with
a sidetrip 4

1 2 5
1 2 3 5 6
3
Touring with 4
4
a detour

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 24

 Infeasible Test Requirements
• An infeasible test requirement cannot be
satisfied
– Unreachable statement (dead code)
• Most testthat
– Subpath criteria
can onlyhave somewith
be executed infeasible test(X > 0
a contradiction
and X < 0)
requirements
• It is usually undecidable whether all test
requirements are feasible
• When sidetrips are not allowed, many
structural criteria have more infeasible test
requirements
• However, always allowing sidetrips weakens
Practical recommendation—Best Effort
the test criteria Touring
– Satisfy as many test requirements as possible
without sidetrips
– Allow
Introduction to Software Testing, sidetrips
Edition 2 (Ch 07) to try to satisfy remaining
© Ammann & Offutt test 25
 Simple & Prime Path Example
‘!’ means path
Simple Len 0 Len 1 Len 2
terminates Len 3
paths [1] [1, 2] [1, 2, 3] [1, 2,‘*’
3, means
4] path
[2] [1, 3] [1, 3, 4] [1, 2, 3, 5]cycles
1 [3] [2, 3] [1, 3, 5] [1, 3, 4, 7] !
[4] [3, 4] [2, 3, 4] [1, 3, 5, 7] !
2 [5] [3, 5] [2, 3, 5] [1, 3, 5, 6] !
[6] [4, 7] ! [3, 4, 7] ! [2, 3, 4, 7] !
[7] ! [5, 7] ! [3, 5, 7] ! [2, 3, 5, 6] !
3
[5, 6] [3, 5, 6] ! [2, 3, 5, 7] !
[6, 5] [5, 6, 5] *
4 5 [6, 5, 7] !
[6, 5, 6] *
6
7 Len 4
[1, 2, 3, 4, 7] !
[1, 2, 3, 5, 7] ! Prime Paths
[1, 2, 3, 5, 6] !
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 26
 Round Trips
• Round-Trip Path : A prime path that starts and
ends at the same node
Simple Round Trip Coverage (SRTC) : TR
contains at least one round-trip path for each
reachable node in G that begins and ends a
round-trip path.
Complete Round Trip Coverage (CRTC) : TR
contains all round-trip paths for each
reachable node in G.
• These criteria omit nodes and edges that are
not in round trips
• Thus, they do not subsume edge-pair, edge, or
node coverage
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 27
 Data Flow Criteria
Goal: Try to ensure that values are computed
and used correctly
• Definition (def) : A location where a value for a
variable is stored into memory
• Use : A location where a variable’s value is
accessed Z = X*2 Defs: def (1) = {X}
2 5 def (5) = {Z}
X=
42
1 4 7 def (6) = {Z}
3 6 Uses: use (5) = {X}
Z = X- use (6) = {X}
8
The values given in defs should reach at least
one, some, or all possible uses
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 28
 DU Pairs and DU Paths
• def (n) or def (e) : The set of variables that are defined by node n
or edge e
• use (n) or use (e) : The set of variables that are used by node n
or
edge e
• DU pair : A pair of locations (li, lj) such that a variable v is
defined at li and used at lj

• Def-clear : A path from li to lj is def-clear with respect to variable

v if v is not given another value on any of the nodes or edges in
the path
• Reach : If there is a def-clear path from li to lj with respect to v,
the def of v at li reaches the use at lj

• du-path : A simple subpath that is def-clear with respect to v

from a def of v to a use of v
• du (ni, nj, v) – the set of du-paths from ni to nj
• du (ni, v) – the set of du-paths that start at ni
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 29
 Touring DU-Paths

• A test path p du-tours subpath d with respect

to v if p tours d and the subpath taken is def-
clear with respect to v

• Sidetrips can be used, just as with previous

touring

• Three criteria
– Use every def
– Get to every use
– Follow all du-paths
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 30
 Data Flow Test Criteria
• First, we make sure every def reaches a use

All-defs coverage (ADC) : For each set of du-

paths S = du (n, v), TR contains at least one
path d in S.
• Then we make sure that every def reaches all
possible uses
All-uses coverage (AUC) : For each set of du-
paths to uses S = du (ni, nj, v), TR contains at
least one path d in S.
• Finally, we cover all the paths between defs
and uses
All-du-paths coverage (ADUPC) : For each set
S = du (ni, nj, v), TR contains every path d in
S.
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 31
 Data Flow Testing Example
Z = X*2
2 5
X = 42
1 4 7
3 6
Z = X-8

All-defs for X All-uses for X All-du-paths for X

[ 1, 2, 4, 5 ] [ 1, 2, 4, 5 ] [ 1, 2, 4, 5 ]
[ 1, 2, 4, 6 ] [ 1, 3, 4, 5 ]
[ 1, 2, 4, 6 ]
[ 1, 3, 4, 6 ]

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 32

 Graph Coverage Criteria
Subsumption Complete
Path
Coverage
CPC
Prime Path
Coverage
PPC
All-DU-
Paths
Coverage Edge-
ADUP Pair
Coverag
e
All-uses Complete
Coverage EPC Round Trip
Edge Coverage
AUC
Coverag CRTC
e
All-defs EC Simple Round
Coverage Trip Coverage
ADC Node SRTC
Coverage
NC
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 33
 Summary 7.1-7.2
• Graphs are a very powerful abstraction for
designing tests
• The various criteria allow lots of cost / benefit
tradeoffs
• These two sections are entirely at the “design
abstraction level” from chapter 2
• Graphs appear in many situations in software
– As discussed in the rest of chapter 7

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 34