Dataflow testing
TDDD04 Lecture 4
Ola Leifler
tisdag 15 april 14
Mariam Kamkar
marka@ida.liu.se
Outline of the Lecture
White box testing
•
Control flow testing
•
Data flow testing*
•
Define / Use Testing
•
Slice-Based Testing
* An Introduction to Data-Flow Testing, Janvi Badlaney, Rohit Ghatol, Romit Jadhwani. ftp://
ftp.csc.ncsu.edu/pub/tech/2006/TR-2006-22.pdf
2
tisdag 15 april 14
Mariam Kamkar
marka@ida.liu.se
Data Flow Testing
Data flow testing focuses on the points at which variables receive values
and the points at which these values are used (or referenced). It detects
improper use of data values (data flow anomalies) due to coding errors.
Rapps and Weyuker’s Motivation*:
“It is our belief that, just as one would not feel confident about a
program without executing every statement in it as part of some test,
one should not feel confident about a program without having seen
the effect of using the value produced by each and every computation.”
* (Data flow analysis techniques for test data selection, sixth international
conference on Software Engineering, Tokyo, Japan, 1982)
3
tisdag 15 april 14
Mariam Kamkar
marka@ida.liu.se
Data Flow Testing (cont.)
Static (compile time, inspection) variants
•
Static analysis techniques
•
Early tools look for data flow anomalies
•
Use before define
•
Define but never used
•
Redefine before use
Dynamic (execution time) variants
•
Look at coverage of data flow properties
4
tisdag 15 april 14
Mariam Kamkar
marka@ida.liu.se
Main categories of data flow coverage criteria
•
Basic blocks (segment of code without branching)
•
All-use
•
C-use (computation use)
•
P-use (predicate use)
•
All-def
•
DU-path (definition-use-path)
5
tisdag 15 april 14
Mariam Kamkar
marka@ida.liu.se
Basic blocks, defs and uses
•
Identify the basic blocks
•
Identify all the definitions
•
•
•
Where variables get their values
Identify all the uses
•
Where variables are used
•
Indicate p-uses (in predicates)
•
Indicate c-uses (in computations)
Draw path from each definition to
each use that might get data from that
definition
6
tisdag 15 april 14
1 s = 0;
2 i = 1;
3 while (i <= n)
{
4
s += i;
5
i ++
}
6 print (s);
7 print (i);
8 print (n);
Mariam Kamkar
marka@ida.liu.se
Basic blocks, defs and uses
•
Identify the basic blocks
•
Identify all the definitions
•
•
•
Where variables get their values
Identify all the uses
•
Where variables are used
•
Indicate p-uses (in predicates)
•
Indicate c-uses (in computations)
Draw path from each definition to
each use that might get data from that
definition
7
tisdag 15 april 14
1 s = 0;
2 i = 1;
3 while (i <= n)
{
4
s += i;
5
i ++
}
6 print (s);
7 print (i);
8 print (n);
Mariam Kamkar
marka@ida.liu.se
Basic blocks, defs and uses
•
Identify the basic blocks
•
Identify all the definitions
•
•
•
Where variables get their values
Identify all the uses
•
Where variables are used
•
Indicate p-uses (in predicates)
•
Indicate c-uses (in computations)
Draw path from each definition to
each use that might get data from that
definition
8
tisdag 15 april 14
Def(s)
1 s = 0;
Def(i)
2 i = 1;
3 while (i <= n)
{
4
s += i; Def(s)
5
i ++
Def(i)
}
6 print (s);
7 print (i);
8 print (n);
Mariam Kamkar
marka@ida.liu.se
Basic blocks, defs and uses
•
Identify the basic blocks
•
Identify all the definitions
•
•
•
Where variables get their values
Identify all the uses
•
Where variables are used
•
Indicate p-uses (in predicates)
•
Indicate c-uses (in computations)
Draw path from each definition to
each use that might get data from that
definition
9
tisdag 15 april 14
1 s = 0;
2 i = 1;
p-use(i),
3 while (i <= n) p-use(n)
{
4
s += i;
c-use(i)
5
i ++
c-use(i)
}
6 print (s);
c-use(s)
7 print (i);
c-use(i)
8 print (n);
c-use(n)
Mariam Kamkar
marka@ida.liu.se
Basic blocks, defs and uses
•
Identify the basic blocks
•
Identify all the definitions
•
•
•
Where variables get their values
Identify all the uses
•
Where variables are used
•
Indicate p-uses (in predicates)
•
Indicate c-uses (in computations)
Draw path from each definition to
each use that might get data from that
definition
10
tisdag 15 april 14
1 s = 0;
2 i = 1;
3 while (i <= n)
{
4
s += i;
5
i ++
}
6 print (s);
7 print (i);
8 print (n);
Mariam Kamkar
marka@ida.liu.se
DEF(x)
Data flow graphs
Data flow diagram
USE(y)
KILL(z)
DEF(x)
USE(x)
USE(z)
KILL(z)
USE(x)
DEF(z)
DEF(y)
USE(z)
USE(y)
USE(z)
KILL(y)
DEF(z)
11
tisdag 15 april 14
•
Control-flow diagram annotated
with define, use and kill
information for each variable
Mariam Kamkar
marka@ida.liu.se
Definitions
•
DEF(v, n): node n in G(P) is a defining node of the variable v in V, iff the
value of the variable v is defined at the statement fragment
corresponding to node n.
•
USE(v, n): node n in G(P) is a usage node of the variable v in V, iff the
value of the variable v is used at the statement fragment corresponding
to node n.
•
P-use, C-use: a usage node USE(v, n) is a predicate use (P-use) iff
statement n is a predicate statement; otherwise, USE(v, n) is a
computation use (C-use).
12
tisdag 15 april 14
Mariam Kamkar
marka@ida.liu.se
View of a single variable
DEF(x)
Procedure
DEF(x)
USE(x)
USE(x)
•
Isolate a single variable
•
Find all paths from start to end
•
Record def-use-kill sequence
Path 1
•
DEF(x), DEF(x), USE(x)
•
Or: ~ddu~
Path 2
13
tisdag 15 april 14
•
DEF(x), USE(x)
•
Or: ~du~
Mariam Kamkar
marka@ida.liu.se
Notation
DEF-USE-KILL
•
d: defined, created, initialized, etc.
•
u: used for something (c: used in computation, p: used in predicate)
•
k: killed, undefined, released
Not in path
•
~d, ~u, ~k: Not in path prior to def, use or kill
•
d~, u~, k~: Not in path after def, use or kill
14
tisdag 15 april 14
Mariam Kamkar
marka@ida.liu.se
Static data flow testing
Look at pairs of letters in the path:
Pair
Verdict
Pair
Verdict
Pair Verdict
dd
ud
kd
du
uu
ku
dk
uk
kk
d~
u~
k~
~d
~u
~k
15
tisdag 15 april 14
Mariam Kamkar
marka@ida.liu.se
View of a single variable
DEF(x)
Path 1: ~ddu~
DEF(x)
USE(x)
USE(x)
•
~d – OK
•
dd – Suspicious
•
du, u~ – OK
Path 2: ~du~
16
tisdag 15 april 14
•
~d – OK
•
du – OK
•
u~ – OK
Mariam Kamkar
marka@ida.liu.se
Quiz!
DEF(x)
For variables y and z
USE(y)
KILL(z)
DEF(x)
USE(x)
USE(z)
KILL(z)
USE(x)
DEF(z)
DEF(y)
USE(z)
USE(y)
USE(z)
KILL(y)
DEF(z)
17
tisdag 15 april 14
•
Draw the diagram
•
Extract all paths
•
Note any problems!
Mariam Kamkar
marka@ida.liu.se
Dynamic data flow testing
Technique: for a particular variable:
•
Find all its definition and usage nodes, then
•
Find the feasible paths among these
•
For each path, devise a “suitable” set of test cases
18
tisdag 15 april 14
Mariam Kamkar
marka@ida.liu.se
Simplified definitions
DU path (with respect to variable v): a path in the annotated control-flow
graph that starts with a vertex containing DEF(v) and ends with a vertex
containing USE(v). Other than the initial vertex, there may be no
occurrences of DEF(v) along the path (it is definition-clear).
Reachable: A USE m of variable v is reachable from a DEF n of variable v,
if and only if there exists a definition-clear path from n to m. (Intuitively, the
value set at the DEF might be used at the USE.)
19
tisdag 15 april 14
Mariam Kamkar
marka@ida.liu.se
Definition-use based testing strategies
•All DU paths: For every feasible DU path in the program, there is a test case that
exercises that path.
•All uses: For every DEF i of variable v and every USE j reachable from i, there is at
least one test case that exercises a DU path from i to j.
•All-p-uses/some-c-uses: For every DEF i of variable v and every P use j
reachable from i, there is at least one test case that exercises a DU path from i to j. If
there is no reachable P use, then there is at least one test case that executes a DU
path from i to a reachable C use of v. (All-c-uses/some-p-uses defined mutatis
mutandis.)
•All-defs: For every DEF of every variable v, there is a test that executes a DU path
from that DEF to a USE of v.
•All-p-uses: For every DEF i of variable v and every P use j reachable from i, there
is at lease one test case that executes a DU path from i to j. (All-c-uses defined
mutatis mutandis.)
20
tisdag 15 april 14
Mariam Kamkar
marka@ida.liu.se
Definition-use based testing strategies
All du paths
DEF(x)
DEF(y)
1
2
•
1-2-3-5-7, 1-2-4-5-7
1-2-4-6-7
All uses
p-USE(y)
•
1-2-3-5-7, 1-2-4-6-7
All-p-uses/some-c-uses
3
4
•
1-2-4-6-7
All-p-uses
5
c-USE(y)
•
c-USE(x)
7
21
tisdag 15 april 14
6
1-2-3-7
Mariam Kamkar
marka@ida.liu.se
Data flow coverage criteria
All paths
Stronger
All DU paths
All uses
All-p/some-c
All-p uses
Branch
Weaker
Statement
22
tisdag 15 april 14
All-c/some-p
All defs
All-c uses
Mariam Kamkar
marka@ida.liu.se
Dynamic data flow testing example
Billing rules
Usage Bill
(minutes)
(Euros)
< 100
40.0 €
101-­‐200
50c per addi5onal minute
> 200
10c per addi5onal minute
23
tisdag 15 april 14
Code
double calculateBill(int usage) {
double bill = 0;
if (usage > 0) {
bill = 40;
}
if (usage > 100) {
if (usage <= 200) {
bill += (usage–100) * 0.5;
}
else {
bill += 50+(usage-200) * 0.1;
if (bill >= 100)
bill = bill * 0.9;
}
}
return bill;
}
1
2
3
4
5
6
def:
usage
def: bill
def: bill
p-use: usage
p-use: usage
7
c-use: usage
c-use: bill
def: bill
p-use: bill
10
c-use: bill
def: bill
11
24
tisdag 15 april 14
Data flow testing example
p-use:
usage
8 c-use: usage
c-use: bill
def: bill
9
Mariam Kamkar
marka@ida.liu.se
c-use: bill
double calculateBill(int usage) {
double bill = 0;
if (usage > 0) {
bill = 40;
}
if (usage > 100) {
if (usage <= 200) {
bill += (usage–100) * 0.5;
}
else {
bill += 50+(usage-200) * 0.1;
if (bill >= 100)
bill = bill * 0.9;
}
}
return bill;
}
1
2
3
4
5
6
def: bill
p-use:
usage
def: bill
p-use: usage
p-use: usage
8 c-use: usage
c-use: bill
def: bill
7
c-use: usage
c-use: bill
def: bill
9
p-use: bill
10
c-use: bill
def: bill
11
25
tisdag 15 april 14
Variable bill
def:
usage
c-use: bill
Paths
•
DU-pairs: (2,11)
•
DU-pairs: (4,11)
•
DU-pairs: (4,7), (7,11)
•
DU-pairs: (4,8), (8,9), (8,11)
•
DU-pairs: (4,8), (8,9), (8,10), (10,11)
Mariam Kamkar
marka@ida.liu.se
1
2
3
4
5
6
def:
usage
def: bill
p-use:
usage
def: bill
p-use: usage
p-use: usage
8 c-use: usage
c-use: bill
def: bill
7
c-use: usage
c-use: bill
def: bill
9
p-use: bill
10
c-use: bill
def: bill
11
26
tisdag 15 april 14
c-use: bill
Mariam Kamkar
marka@ida.liu.se
Variable bill
All DU paths
•
2-3-5-11
•
4-5-11
•
4-5-6-7
•
4-5-6-8
•
7-11
•
8-9
•
8-9-10
•
8-9-11
•
10-11
1
2
3
4
5
6
def:
usage
def: bill
Path
def: bill
p-use: usage
2-­‐3-­‐5-­‐11
4-­‐5-­‐11
4-­‐5-­‐6-­‐7
p-use: usage
4-­‐5-­‐6-­‐8
7-­‐11
8-­‐9
7
c-use: usage
c-use: bill
def: bill
p-use: bill
Test cases for variable bill
p-use:
usage
8 c-use: usage
c-use: bill
def: bill
9
Mariam Kamkar
marka@ida.liu.se
8-­‐9-­‐10
8-­‐9-­‐11
10-­‐11
10
c-use: bill
def: bill
11
27
tisdag 15 april 14
c-use: bill
usage
Expected
Actual
1
2
3
4
5
6
def:
usage
Mariam Kamkar
marka@ida.liu.se
Test cases for variable bill
def: bill
p-use:
usage
usage > 0
Path
def: bill
p-use: usage
p-use: usage
8 c-use: usage
c-use: bill
def: bill
7
c-use: usage
c-use: bill
def: bill
9
p-use: bill
bill >= 100
10
c-use: bill
def: bill
11
28
tisdag 15 april 14
c-use: bill
usage
2-­‐3-­‐5-­‐11 0
usage > 100
usage <= 200
Expected
Actual
0.0
0.0
4-­‐5-­‐11 50
40.0
40.0
4-­‐5-­‐6-­‐7 140
60.0
60.0
4-­‐5-­‐6-­‐8 220
92.0
92.0
7-­‐11
140
60.0
60.0
8-­‐9
400
110.0
99.0
8-­‐9-­‐10 220
92.0
92.0
8-­‐9-­‐11 400
110.0
99.0
10-­‐11
92.0
92.0
220
1
2
3
4
def:
usage
Mariam Kamkar
marka@ida.liu.se
Variable usage
def: bill
p-use:
usage
usage > 0
def: bill
Is 3-5-6 valid?
5
6
p-use: usage
p-use: usage
8 c-use: usage
c-use: bill
def: bill
7
c-use: usage
c-use: bill
def: bill
9
p-use: bill
bill >= 100
10
c-use: bill
def: bill
11
p-use: bill
29
tisdag 15 april 14
usage > 100
usage <= 200
DU pairs
•
(1,3)
•
(1,5)
•
(1,6)
•
(1,7)
•
(1,8)
1
2
3
4
5
6
def:
usage
p-use:
usage
p-use: usage
p-use: usage
p-use: bill
30
tisdag 15 april 14
usage
Expected
Actual
def: bill
10
c-use: bill
def: bill
11
usage > 0
Path
7
c-use: usage
c-use: bill
def: bill
p-use: bill
Test cases for variable usage
def: bill
8 c-use: usage
c-use: bill
def: bill
9
Mariam Kamkar
marka@ida.liu.se
usage > 100
usage <= 200
1-­‐2-­‐3
50
40.0
40.0
1-­‐2-­‐3-­‐4-­‐5
220
92.0
92.0
1-­‐2-­‐3-­‐4-­‐5-­‐6
140
60.0
60.0
1-­‐2-­‐3-­‐4-­‐5-­‐6-­‐7
140
60.0
60.0
1-­‐2-­‐3-­‐4-­‐5-­‐6-­‐8
220
92.0
92.0
Mariam Kamkar
marka@ida.liu.se
Applicability and limitations
•
It should be used for all modules of code that cannot be tested
sufficiently through reviews and inspections.
•
Tester must have sufficient programming skill
•
Can be very time consuming
31
tisdag 15 april 14
Mariam Kamkar
marka@ida.liu.se
WRAP-UP
32
tisdag 15 april 14
Mariam Kamkar
marka@ida.liu.se
Data flow testing summary
•
Based on data flow analysis of program under test
•
Possibly better at finding bugs than structural testing
•
Static and dynamic testing
•
•
Static testing (compile-time) – look for anomalies in e.g. def-use-kill patterns
•
Dynamic testing – create test cases to cover specific flows
Other forms of data flow testing
•
33
tisdag 15 april 14
Slice-based testing