Overview Graph Coverage Criteria
( Introduction to Software Testing
Chapter 2.1, 2.2)
Paul Ammann & Jeff Offutt
Hierarchy of Structural/graph SW Coverages
Complete Value
Coverage
CVC
Complete Path
Coverage
CPC
All-DU-Paths
Coverage
ADUP
All-uses
Coverage
AUC
All-defs
Coverage
ADC
(SW) Model checking
Concolic testing
Prime Path
Coverage
PPC
Edge-Pair
Coverage
EPC
Edge
Coverage
EC
Node
Coverage
NC
Complete Round Trip
Coverage
CRTC
Simple Round Trip
Coverage
SRTC
2/60
Covering Graphs (2.1)

Graphs are the most commonly used structure for testing

Graphs can come from many sources





Control flow graphs
Design structure
FSMs and statecharts
Use cases
Tests usually are intended to “cover” the graph in some way
3
Definition of a Graph

A set N of nodes, N is not empty

A set N0 of initial nodes, N0 is not empty

A set Nf of final nodes, Nf is not empty

A set E of edges, each edge from one node to another

( ni , nj ), i is predecessor, j is successor
4
Three Example Graphs
0
0
1
2
3
3
1
4
7
2
5
8
0
6
9
1
Not a
valid
graph
2
3
N0 = { 0 }
N0 = { 0, 1, 2 }
N0 = { }
Nf = { 3 }
Nf = { 7, 8, 9 }
Nf = { 3 }
5
Paths in Graphs
Path : A sequence of nodes – [n1, n2, …, nM]


Each pair of nodes is an edge
Length : The number of edges


A single node is a path of length 0
Subpath : A subsequence of nodes in p is a subpath of p
Reach (n) : Subgraph that can be reached from n


0
1
2
Paths
[ 0, 3, 7 ]
3
4
5
6
[ 1, 4, 8, 5, 1 ]
[ 2, 6, 9 ]
7
8
Reach (0) = { 0, 3, 4,
7, 8, 5, 1, 9 }
Reach ({0, 2}) = G
Reach([2,6]) = {2, 6,
9}
9
6
Test Paths and SESEs


Test Path : A path that starts at an initial node and ends at a
final node
Test paths represent execution of test cases



Some test paths can be executed by many tests
Some test paths cannot be executed by any tests
SESE graphs : All test paths start at a single node and end
at another node


Single-entry, single-exit
N0 and Nf have exactly one node
1
0
4
3
2
6
5
Double-diamond graph
Four test paths
[ 0, 1, 3, 4, 6 ]
[ 0, 1, 3, 5, 6 ]
[ 0, 2, 3, 4, 6 ]
[ 0, 2, 3, 5, 6 ]
7
Visiting and Touring


Visit : A test path p visits node n if n is in p
A test path p visits edge e if e is in p
Tour : A test path p tours subpath q if q is a subpath of p
Path [ 0, 1, 3, 4, 6 ]
Visits nodes 0, 1, 3, 4, 6
Visits edges (0, 1), (1, 3), (3, 4), (4, 6)
Tours subpaths (0, 1, 3), (1, 3, 4), (3, 4, 6), (0, 1, 3, 4), (1, 3, 4, 6)
8
Tests and Test Paths

path (t) : The test path executed by test t

path (T) : The set of test paths executed by the set of tests T

Each test executes one and only one test path
A location in a graph (node or edge) can be reached from
another location if there is a sequence of edges from the first
location to the second



Syntactic reach : A subpath exists in the graph
Semantic reach : A test exists that can execute that subpath
9
Tests and Test Paths
test 1
many-to-one
Test
Path
test 2
test 3
Deterministic software – a test always executes the same test path
test 1
many-to-many
Test Path 1
test 2
Test Path 2
test 3
Test Path 3
Non-deterministic software – a test can execute different test paths
10
Testing and Covering Graphs (2.2)

We use graphs in testing as follows :


Developing a model of the software as a graph
Requiring tests to visit or tour specific sets of nodes, edges or subpaths
• Test Requirements (TR) : Describe properties of test paths
• Test Criterion : Rules that define test requirements
• Satisfaction : Given a set TR of test requirements for a criterion C,
a set of tests T satisfies C on a graph if and only if for every test
requirement in TR, there is a test path in path(T) that meets the test
requirement tr
• Structural Coverage Criteria : Defined on a graph just in terms
of nodes and edges
• Data Flow Coverage Criteria : Requires a graph to be annotated
with references to variables
11
Node and Edge Coverage

Edge coverage is slightly stronger than node coverage
• The “length up to 1” allows for graphs with one node and
no edges
• NC and EC are only different when there is an edge and
another subpath between a pair of nodes (as in an “ifelse” statement)
0
1
2
Node Coverage : TR = { 0, 1, 2 }
Test Path = [ 0, 1, 2 ]
Edge Coverage : TR = { (0,1), (0, 2), (1, 2) }
Test Paths = [ 0, 1, 2 ]
[ 0, 2 ]
12
Paths of Length 1 and 0

A graph with only one node will not have any edges
0
• It may be boring, but formally, Edge Coverage needs to
require Node Coverage on this graph
• Otherwise, Edge Coverage will not subsume Node
Coverage
– So we define “length up to 1” instead of simply “length 1”
• We have the same issue with graphs that only
have one edge – for Edge Pair Coverage …
0
1
13
Covering Multiple Edges

Edge-pair coverage requires pairs of edges, or subpaths of
length 2
• The “length up to 2” is used to include graphs that have
less than 2 edges
• The logical extension is to require all paths …
• Unfortunately, this is impossible if the graph has a loop, so a
weak compromise is to make the tester decide which paths:
14
Structural Coverage Example
Node Coverage
TRNC = { 0, 1, 2, 3, 4, 5, 6 }
Test Paths: [ 0, 1, 2, 3, 6 ] [ 0, 1, 2, 4, 5, 4, 6 ]
0
Edge Coverage
TREC ={(0,1),(0,2),(1,2), (2,3), (2,4), (3,6), (4,5),(4,6), (5,4)}
Test Paths: [ 0, 1, 2, 3, 6 ] [ 0, 2, 4, 5, 4, 6 ]
1
Edge-Pair Coverage
TREPC = { [0,1,2], [0,2,3], [0,2,4], [1,2,3], [1,2,4], [2,3,6],
[2,4,5], [2,4,6], [4,5,4], [5,4,5], [5,4,6] }
Test Paths: [ 0, 1, 2, 3, 6 ] [ 0, 1, 2, 4, 6 ] [ 0, 2, 3, 6 ]
[ 0, 2, 4, 5, 4, 5, 4, 6 ]
2
3
4
5
6
Complete Path Coverage
Test Paths: [ 0, 1, 2, 3, 6 ] [ 0, 1, 2, 4, 6 ] [ 0, 1, 2, 4, 5, 4, 6 ]
[ 0, 1, 2, 4, 5, 4, 5, 4, 6 ] [ 0, 1, 2, 4, 5, 4, 5, 4, 5, 4, 6 ] …
15
Loops in Graphs

If a graph contains a loop, it has an infinite number of
paths

Thus, CPC is not feasible

SPC is not satisfactory because the results are
subjective and vary with the tester

Attempts to “deal with” loops:



1980s : Execute each loop, exactly once ([4, 5, 4] in previous example)
1990s : Execute loops 0 times, once, more than once
2000s : Prime paths
16
Simple Paths and Prime Paths

Simple Path : A path from node ni to nj is simple, if no node
appears more than once, except possibly the first and last
nodes are the same




No internal loops
Includes all other subpaths
A loop is a simple path
Prime Path : A simple path that does not appear as a proper
subpath of any other simple path
0
1
2
Simple Paths : [ 0, 1, 3, 0 ], [ 0, 2, 3, 0], [ 1, 3, 0, 1 ],
[ 2, 3, 0, 2 ], [ 3, 0, 1, 3 ], [ 3, 0, 2, 3 ], [ 1, 3, 0, 2 ],
[ 2, 3, 0, 1 ], [ 0, 1, 3 ], [ 0, 2, 3 ], [ 1, 3, 0 ], [ 2, 3, 0 ],
[ 3, 0, 1 ], [3, 0, 2 ], [ 0, 1], [ 0, 2 ], [ 1, 3 ], [ 2, 3 ], [ 3, 0 ],
[0], [1], [2], [3]
3
Prime Paths : [ 0, 1, 3, 0 ], [ 0, 2, 3, 0], [ 1, 3, 0, 1 ],
[ 2, 3, 0, 2 ], [ 3, 0, 1, 3 ], [ 3, 0, 2, 3 ], [ 1, 3, 0, 2 ],
[ 2, 3, 0, 1 ]
17
Prime Path Coverage

A simple, elegant and finite criterion that requires loops to be
executed as well as skipped
• Will tour all paths of length 0, 1, …
• That is, it subsumes node, edge, and edge-pair coverage
18
Prime Path Example


The previous example has 38 simple paths
Only nine prime paths
0
1
2
3
4
5
6
Prime Paths
[ 0, 1, 2, 3, 6 ]
[ 0, 1, 2, 4, 5 ]
[ 0, 1, 2, 4, 6 ]
[ 0, 2, 3, 6 ]
[ 0, 2, 4, 5]
[ 0, 2, 4, 6 ]
[ 5, 4, 6 ]
[ 4, 5, 4 ]
[ 5, 4, 5 ]
Execute
loop 0 times
Execute
loop once
Execute loop
more than once
19
Simple & Prime Path Example
Simple
paths
0
1
2
3
Len 0
[0]
[1]
[2]
[3]
[4]
[5]
[6] !
Len 1
[0, 1]
[0, 2]
[1, 2]
[2, 3]
[2, 4]
[3, 6] !
[4, 6] !
[4, 5]
[5, 4]
4
5
6
Len 4
[0, 1, 2, 3, 6] !
[0, 1, 2, 4, 6] !
[0, 1, 2, 4, 5] !
‘!’ means path
Len
2
Len 3
terminates
[0, 1, 2]
[0, 1,‘*’
2, means
3]
path
[0, 2, 3]
[0, 1, 2, 4]cycles
[0, 2, 4]
[0, 2, 3, 6] !
[1, 2, 3]
[0, 2, 4, 6] !
[1, 2, 4]
[0, 2, 4, 5] !
[2, 3, 6] !
[1, 2, 3, 6] !
[2, 4, 6] !
[1, 2, 4, 5] !
[2, 4, 5] !
[1, 2, 4, 6] !
[4, 5, 4] *
[5, 4, 6] !
[5, 4, 5] *
Prime Paths
Note that paths w/o ! or * cannot be prime paths
20
Weaknesses of the Purely Structural Coverage
/* TC1: x= 1, y= 1;
TC2: x=-1, y=-1;*/
void foo(int x, int y) {
if ( x > 0)
x++;
else
x--;
if(y >0)
y++;
else
y--;
assert (x * y >= 0);
}
yes
x>
0
x++
no
x--
y>
0
y++
y--
assert(x*y>=0
)
Purely structural coverage (e.g., branch coverage) alone
cannot improve the quality of target software sufficiently
-> Advanced semantic testing should be accompanied
21
Final Remarks
1. Why are coverage criteria important for testing?
2. Why is branch coverage popular in industry?
3. Why is prime path coverage not use in practice?
4. Why is it difficult to reach 100% branch coverage of
real-world programs?
22