DF14900
Software Testing
White Box Testing
"
!
!
Peter Bunus (Lecturer), Mariam Kamkar!
Department of Computer and Information Science!
Linköping University, Sweden!
peter.bunus@liu.se, mariam.kamkar@liu.se!
Outline of the Lecture"
§  White box testing (Glass box testing, Open box testing, Clear box testing, Structural
testing)!
Ÿ  Control flow testing"
Ÿ  Data flow testing"
Part II"
Unit Testing: White box testing!
!
2
White box testing: control flow testing"
§
§
§
§
§
logical decision!
loops!
internal data structure!
paths!
...!
Coverage!!
Part II"
Unit Testing: White box testing!
!
3
White box testing: control flow testing"
§  Definition: a strategy in which testing is based on the internal paths,
structure, and implementation of the software under test (SUT)
!
§  Applicability: all levels of system development (path testing!)!
Ÿ
Ÿ
Ÿ
Ÿ
Unit!
Integration!
System!
Acceptance
!
§  Disadvantages: 1) number of execution paths may be so large; 2) test
cases may not detect data sensitivity; 3) assumes that control flow is
correct (nonexistent paths!); 4) tester must have programming skills.
!
§  Advantages: tester can be sure that every path have been identified
and tested.!
Part II"
Unit Testing: White box testing!
!
4
Control Flow Graphs"
Process blocks
5
Decision Point
Junction Point
Sequence
If
While
Part II"
Unit Testing: White box testing!
!
Until
Case
6
!Definition: Given a program written in an imperative
programming language, its program graph is a
directed graph in which nodes are statement
fragments, and edges represent flow of control (a
complete statement is a “default” statement
fragment).!
Part II"
Unit Testing: White box testing!
!
Code Coverage (test coverage metrics)"
Levels of Coverage:!
!
§  Statement/Line/Basic block/Segment Coverage!
§  Decision (Branch) Coverage!
§  Condition Coverage!
§  Multiple Condition Coverage!
§  Decision/Condition Coverage!
§  Loop Coverage!
§  Path Coverage!
Part II"
Unit Testing: White box testing!
!
7
Statement Coverage"
Student worksheet !
Begin!
if ( y >= 0) !
!
!then y = 0;!
abs = y;!
end;!
8
begin!
y >= 0!
y = 0!
abs = y!
Write test cases!!
Part II"
Unit Testing: White box testing!
!
Statement Coverage"
Begin!
if ( y >= 0) !
!
!then y = 0;!
abs = y;!
end;!
9
begin!
y >= 0!
y = 0!
abs = y!
test case-1(yes):"
•  input:
! y = ?!
•  expected result: !?!
•  actual result:
!?!
Part II"
Unit Testing: White box testing!
!
yes!
Statement Coverage"
Begin!
if ( y >= 0) !
!
!then y = 0;!
abs = y;!
end;!
10
begin!
y >= 0!
y = 0!
abs = y!
test case-1(yes):"
•  input:
! y = 0!
•  expected result: !0!
•  actual result:
!0!
Part II"
Unit Testing: White box testing!
!
yes!
What is Wrong with Line Coverage
Steve Cornett (Bullseye testing technology)"
!Software developers and testers commonly use line coverage because
of its simplicity and availability in object code instrumentation
technology. Of all the structural coverage criteria, line coverage is the weakest,
indicating the fewest number of test cases. Bugs can easily occur in the cases that line coverage cannot see. !
!The most significant shortcoming of line coverage is that it fails to
measure whether you test simple if statements with a false decision
outcome. Experts generally recommend to only use line coverage if
nothing else is available. Any other measure is better.!
Part II"
Unit Testing: White box testing!
!
11
Decision (Branch) Coverage"
Student worksheet !
Begin!
if ( y >= 0) !
!
!then y = 0;!
abs = y;!
end;!
begin!
y >= 0!
no!
abs = y!
Part II"
Unit Testing: White box testing!
!
12
yes!
y = 0!
Decision (Branch) Coverage"
Begin!
if ( y >= 0) !
!
!then y = 0;!
abs = y;!
end;!
13
begin!
y >= 0!
no!
yes!
y = 0!
abs = y!
test case-1(yes):"
•  input:
! y = 0!
•  expected result: !0!
•  actual result:
!0!
Part II"
Unit Testing: White box testing!
!
test case-2(no):"
•  input:
! y = ?!
•  expected result: !?!
•  actual result:
!?!
Decision (Branch) Coverage"
Begin!
if ( y >= 0) !
!
!then y = 0;!
abs = y;!
end;!
14
begin!
y >= 0!
no!
yes!
y = 0!
abs = y!
test case-1(yes):"
•  input:
! y = 0!
•  expected result: !0!
•  actual result:
!0!
Part II"
Unit Testing: White box testing!
!
test case-2(no):"
•  input:
! y = -5!
•  expected result: ! 5!
•  actual result:
!-5!
… more conditions?"
Condition Coverage"
16
Student worksheet !
Begin!
if ( x < 10 && y > 20) {
z = foo (x,y); else z =fie
(x,y);!
}!
end;!
Part II"
Unit Testing: White box testing!
!
!!
x<10
&&
y>20!
no"
z=fie (x,y)
!
yes"
z=foo (x,y)
Condition Coverage"
Begin!
if ( x < 10 && y > 20) {
z = foo (x,y); else z =fie
(x,y);!
}!
end;!
test case-1(T,F):"
•  input:
x= ? y=?!
•  expected result:
?!
•  actual result:
?!
Part II"
Unit Testing: White box testing!
!
!!
17
x<10
&&
y>20!
no"
z=fie (x,y)
!
test case-2 (F,T):"
•  input:
x = ? y = ?!
•  expected result:
?!
•  actual result:
?!
yes"
z=foo (x,y)
Condition Coverage"
Begin!
if ( x < 10 && y > 20) {
z = foo (x,y); else z =fie
(x,y);!
}!
end;!
!!
test case-1(T,F):"
•  input:
x = -4 y = 12!
•  expected result:
?!
•  actual result:
?!
Part II"
Unit Testing: White box testing!
!
18
x<10
&&
y>20!
no"
z=fie (x,y)
!
yes"
z=foo (x,y)
test case-2 (F;T):"
•  input:
x = 12 y = 30!
•  expected result:
?!
•  actual result:
?!
Problems?
in this example taking only false condition (”no”
branch) in control flow
What about decision / condition"
20
Multiple Condition Coverage"
Student worksheet !
Begin!
if ( x < 10 && y > 20) {
z = foo (x,y); else z =fie
(x,y);!
}!
end;!
Part II"
Unit Testing: White box testing!
!
!!
no"
x < 10!
no"
yes"
y > 20!
yes"
z=fie (x,y) !
z=foo (x,y) !
21
Multiple Condition Coverage"
Begin!
if ( x < 10 && y > 20) {
z = foo (x,y); else z =fie
(x,y);!
}!
end;!
!!
!
x < ? y > ?!
-----------------------------------!
•  test-case-1:
!t
!t!
•  test-case:2 !
!t
!f!
•  test-case-3:!
!f
!t!
•  test-case-4 !
!f
!f!
Part II"
Unit Testing: White box testing!
!
no"
x < 10!
no"
yes"
y > 20!
yes"
z=fie (x,y) !
z=foo (x,y) !
22
Multiple Condition Coverage"
Begin!
if ( x < 10 && y > 20) {
z = foo (x,y); else z =fie
(x,y);!
}!
end;!
!!
!
x < 10 y > 20!
-----------------------------------!
•  test-case-1:
!t
!t!
•  test-case:2 !
!t
!f!
•  test-case-3:!
!f
!t!
•  test-case-4 !
!f
!f!
Part II"
Unit Testing: White box testing!
!
no"
x < 10!
no"
yes"
y > 20!
yes"
z=fie (x,y) !
z=foo (x,y) !
Decision/Condition Coverage"
23
Student worksheet !
Begin!
if ( x < 10 && y > 20) {
z = foo (x,y); else z =fie
(x,y);!
}!
end;!
Part II"
Unit Testing: White box testing!
!
no"
!!
z=fie (x,y) !
x<10!
&&!
y>20!
yes"
z=foo (x,y) !
Decision/Condition Coverage"
Begin!
if ( x < 10 && y > 20) {
z = foo (x,y); else z =fie
(x,y);!
}!
end;!
!!
test case-1 (T,T,yes):"
•  input:
x = ? y = ?!
•  expected result:
?!
•  actual result:
?!
Part II"
Unit Testing: White box testing!
!
no"
z=fie (x,y) !
24
x<10!
&&!
y>20!
yes"
z=foo (x,y) !
test case-2 (F,F,no):"
•  input:
x = ? y = ?!
•  expected result:
?!
•  actual result:
?!
Decision/Condition Coverage"
Begin!
if ( x < 10 && y > 20) {
z = foo (x,y); else z =fie
(x,y);!
}!
end;!
!!
test case-1 (T,T,yes):"
•  input:
x = -4 y = 30!
•  expected result:
?!
•  actual result:
?!
Part II"
Unit Testing: White box testing!
!
no"
z=fie (x,y) !
25
x<10!
&&!
y>20!
yes"
z=foo (x,y) !
test case-2 (F,F,no):"
•  input:
x = 12 y = 12!
•  expected result:
?!
•  actual result:
?!
Loop Coverage"
26
first!
first!
first!
first!
A
A
A
B
B
B
C
C
C
D
D
D
last!
last!
last!
A
B
C
D
last!
Simple"
Concatenated"
Part II"
Unit Testing: White box testing!
!
Nested"
Knotted (horrible)/"
Unstructured "
Loop Coverage "
§  Simple loops :"
Ÿ
Ÿ
Ÿ
Ÿ
Skip the loop entirely !
Only one pass through the loop!
Two passes through the loop!
m passes through the loop when m < n m: small number representing a typical loop value n: maximum number of allowable passes through the loop!
Ÿ  n-1, n, n+1 passes through the loop
!
§  Nested loops:"
Ÿ  Start at the innermost loop, set all other loops to minimum value.!
Ÿ  Conduct Simple loop test for innermost loop. Add other tests for
out-of-range or excluded values.!
Ÿ  Continue until all loops have been tested.!
Part II"
Unit Testing: White box testing!
!
27
Loop Coverage "
§  Concatenated loops :"
Ÿ  If each of the loops is independent of the other, use Simple
loops approach!
Ÿ  If the loops are concatenated and dependent ( counter for
loop 1 is used as the initial value for loop 2) then use Nested
loop approach.!
§  Knotted (horrible) /unstructured loops:"
Ÿ  Whenever possible, this class of loops should be redesigned
to reflect the use of the structured programming constructs. !
Part II"
Unit Testing: White box testing!
!
28
Path Coverage "
§  A path is a sequence of branches, or conditions.!
!
§  A path corresponds to a test case, or a set of inputs.!
§  In code coverage testing, branches have more
importance than the blocks they connect.!
§  Bugs are often sensitive to branches and conditions.!
Part II"
Unit Testing: White box testing!
!
29
Path with loops "
30
a"
d"
b"
c"
a"
e"
?"
?"
e"
Part II"
Unit Testing: White box testing!
!
Path with loops "
31
a"
d"
b"
c"
a"
e"
c,b,d"
d"
e"
Part II"
Unit Testing: White box testing!
!
Path Coverage (cont.)"
§  All possible execution paths
!
§  Question: How do we know how many paths to look for?
!
§  Answer: The computation of cyclomatic complexity !
Part II"
Unit Testing: White box testing!
!
32
Cyclomatic Complexity (McCabeʼs Metric)"
!Cyclomatic
Complexity is a software metric that
provides a quantitative measure of the logical complexity
of a program. When used in context of the basis path
testing method, the value computed for cyclomatic
complexity defines the number of independent paths in
the basis set of a program and provides us with an upper
bound for the number of tests that must be conducted to
ensure that all statements have been executed at least
once.!
Part II"
Unit Testing: White box testing!
!
33
Computation of cyclomatic complexity"
!Cyclomatic
1.
2.
complexity has a foundation in graph theory and is
computed in the following ways:
!
Cyclomatic complexity V(G), for a flow graph, G, is defined as:
!
!
!
!V(G) = E – N + 2P"
!
!
!E: number of edges!
!
!
!N: number of nodes!
!
!
!P: number of disconnected parts of the graph!
Cyclomatic complexity V(G), for a flow graph, G, with only binary
decisions, is defined as:!
! !
! !
!
!
!V(G) = b + 1"
!b: number of binary decision!
Part II"
Unit Testing: White box testing!
!
34
Examples of Graphs and calculation of McCabeʼs Complexity Metric"
Student worksheet !
E= , N= , P=
V=
E= , N= , P=
V=
E= , N= , P=
V=
E= , N= , P=
V=
E=, N= , P=
V=
Part II"
Unit Testing: White box testing!
!
35
Examples of Graphs and calculation of McCabeʼs Complexity Metric"
E= 1, N= 2, P= 1
V = 1-2+2=1
E= 4, N= 4, P= 1
V= 4-4+2= 2
E= 2, N= 4, P= 2
V= 2-4+2*2= 2
E= 12, N= 7, P= 1
V= 12-7+2= 7
E= 13, N= 11, P= 3
V= 13-11+2*3= 8
Part II"
Unit Testing: White box testing!
!
36
Complexity Sum and Sum of Complexities"
E1, N1!
E2, N2!
V1= E1-N1+2"
V2= E2-N2+2"
Part II"
Unit Testing: White box testing!
!
37
E1, N1!
E2, N2!
V1+V2= E1+E2-N1-N2+2*2 = V1 + V2"
38
A
C
B
D
F
E
H
G
L
K
P
O
N
M
J
I
R
Q
S
Part II"
Unit Testing: White box testing!
!
1. V(G) = E – N + 2P"
!
E = ?!
N=?!
P = ?!
V(G) = ?!
!
!
2.  V(G) = b + 1"
b = ?!
V(G) = ?!
39
A
C
B
D
F
E
H
G
L
K
P
O
N
M
J
I
R
Q
S
Part II"
Unit Testing: White box testing!
!
1. V(G) = E – N + 2P"
!
E = 24!
N = 19!
P = 1!
V(G) = 24-19+2 = 7!
!
!
2.  V(G) = b + 1"
b = 6!
V(G) = 6+1 = 7!
Independent Paths"
!An independent path is any path through the
program that introduces at least one new set of
processing statements or a new condition. When
stated in terms of a flow graph, an independent path
must move along at least one edge that has not
been traversed before the path is defined.!
Part II"
Unit Testing: White box testing!
!
40
Basis Path Testing"
1.  Derive the control flow graph from the software module.
!
2.  Compute the graphʼs Cyclomatic Complexity of the
resultant flow graph.
!
3.  Determine a basis set of linearly independent paths.*"
4.  Create a test case for each basis path.
!
5.  Execute these tests.!
* We will go through this step now.!
!
Part II"
Unit Testing: White box testing!
!
41
Determine a basis set of linearly independent paths
McCabeʼs baseline method"
1.
2.
3.
4.
5.
Pick a “baseline” path. This path should be a “normal case”
program execution. McCabe advises: choose a path with as
many decision as possible.
"
To choose the next path, change the outcome of the first
decision along the baseline path while keeping the maximum
number of other decisions the same as the baseline path. !
To generate the third path, begin again with the baseline but
vary the second decision rather than the first.
!
Repeat the 3 for other paths until all decision along baseline
path have been flipped.
!
Now proceed to the second path, flipping its decisions, one by
one until the basis path set is completed.!
Part II"
Unit Testing: White box testing!
!
42
Path-2
Path-1
1:a decision"
A
2:a decision"
3:e decision"
R
Q
ABDEGKMQS
S
Part II"
Unit Testing: White box testing!
!
H
G
L
P
O
N
M
J
I
K
P
O
F
E
J
I
N
C
D
6:e decision" L
4:e decision" K
A 1:a decision"
B
D
H
M
Find 7 Paths"
5:decision" F
E
G
Student worksheet !
C
B
43
R
Q
S
A
?
Path-1
Path-2
44
1:a decision A
A 1:a decision"
C
B
2:a decision D
3:e decision
H
G
R
Q
S
Part II"
Unit Testing: White box testing!
!
H
G
L
P
O
N
M
J
I
K
P
O
N
ABDEGKMQS
J
I
F
E
6:e decision L
4:e decision K
M
D
5:decision F
E
C
B
R
Q
S
ACDEGKMQS
Path-1
Path-3
A
2:e decision"
Student worksheet !
C
B
R
Q
ABDEGKMQS!
S
Part II"
Unit Testing: White box testing!
!
L
P
O
N
M
J
I
K
P
O
F
H
G
L
N
2:e decision"
E
J
I
K
M
C
D
F
H
A
B
D
E
G
45
R
Q
S
A
?
Path-3
Path-1
46
A
A
C
B
2:e decision"
D
D
F
E
H
G
R
Q
ABDEGKMQS
S
Part II"
Unit Testing: White box testing!
!
L
P
O
N
M
J
I
K
P
O
F
H
G
L
N
2:e decision"
E
J
I
K
M
C
B
R
Q
S
ABDFILORS
Path-4
Path-1
A
Student worksheet !
C
B
F
R
Q
ABDEGKMQS!
S
Part II"
Unit Testing: White box testing!
!
L
P
O
N
M
J
I
K
P
O
H
G
L
N
F
E 3:e decision"
J
I
K
M
C
D
E
H
A
B
D
3:e decision"
G
47
R
Q
S
A
?
Path-4
Path-1
48
A
A
C
B
D
3:e decision"
D
F
E
H
G
R
Q
ABDEGKMQS
S
Part II"
Unit Testing: White box testing!
!
L
P
O
N
M
J
I
K
P
O
H
G
L
N
F
E 3:e decision"
J
I
K
M
C
B
R
Q
S
ABDEHKMQS
Path-1
Path-5
Student worksheet !
A
C
B
49
D
F
E
H
4:e decision"
R
Q
ABDEGKMQS!
S
Part II"
Unit Testing: White box testing!
!
L
P
O
N
M
J
I
K 4:e decision!
P
O
H
G
L
N
F
E
J
I
K
M
C
B
D
G
A
R
Q
S
A
?
Path-5
Path-1
50
A
A
C
B
D
D
F
E
H
G
4:e decision"
R
Q
ABDEGKMQS
S
Part II"
Unit Testing: White box testing!
!
L
P
O
N
M
J
I
K 4:e decision"
P
O
H
G
L
N
F
E
J
I
K
M
C
B
R
Q
S
ABDEGKNQS
Path-6
Path-2
A
Student worksheet !
C
B
51
D
5:e decision" F
H
G
R
Q
ACDEGKMQS
S
Part II"
Unit Testing: White box testing!
!
L
P
O
N
M
J
I
K
P
O
H
G
L
N
5:e decision" F
E
J
I
K
M
C
B
D
E
A
R
Q
S
A
?
Path-2
Path-6
52
A
A
C
B
D
H
G
D
5:e decision" F
E
R
Q
ACDEGKMQS
S
Part II"
Unit Testing: White box testing!
!
L
P
O
N
M
J
I
K
P
O
H
G
L
N
5:e decision" F
E
J
I
K
M
C
B
R
Q
S
ACDFJLORS
Path-2
Path-7
A
Student worksheet !
C
B
53
D
F
E
H
R
Q
ACDEGKMQS
S
Part II"
Unit Testing: White box testing!
!
H
G
P
O
N
M
J
I
6:e decision! L
K
P
O
N
M
J
I
F
E
6:e decision" L
K
C
B
D
G
A
R
Q
S
A
?
Path-7
Path-2
54
A
A
C
B
D
D
F
E
H
G
ACDEGKMQS
S
Part II"
Unit Testing: White box testing!
!
P
O
N
M
J
I
6:e decision" L
K
P
R
Q
H
G
L
O
N
M
F
E
J
I
6:e decision"
K
C
B
R
Q
S
ACDFILPRS
55
Set of basis paths:
A
C
B
D
F
E
H
G
L
K
P
O
N
M
J
I
R
Q
S
Part II"
Unit Testing: White box testing!
!
1.  A
?
2.  A
?
3.  A
?
4.  A
?
5.  A
?
6.  A
?
7.  A
?
56
Set of basis paths:
A
C
B
1.  ABDEGKMQS
D
2.  ACDEGKMQS
F
E
H
G
4.  ABDEHKMQS
L
K
P
O
N
M
J
I
R
Q
S
Part II"
Unit Testing: White box testing!
!
3.  ABDFILORS
5.  ABDEGKNQS
6.  ACDFJLORS
7.  ACDFILPRS
Observation"
§  Basis path testing calls for the creation of a test
case for each of these paths.
!
§  This set of test cases will guarantee both statement
and branch coverage.
"
§  Note that multiple sets of basis paths can be created
that are not necessarily unique. Each set, however,
has the property that a set of test cases based on it
will execute every statement and every branch.!
Part II"
Unit Testing: White box testing!
!
57
Guidelines"
§  Program with high cyclomatic complexity require more testing. !
§  Of the organizations that use the cyclomatic complexity metric,
most set some guideline for maximum acceptable complexity; V
(G) =10 is a common choice. !
§  What happens if a unit has higher complexity?!
Ÿ  Either simplify the unit"
Ÿ  Or plan to do more testing"
§  In general, when a program is well structured (i.e., composed
solely of the structured programming constructs), it can be
reduced to a graph with one path.
!
§  If the unit is well structured, its essential complexity is 1"
!
Part II"
Unit Testing: White box testing!
!
58
Structured programming constructs"
Sequence!
If-Then!
Part II"
Unit Testing: White box testing!
!
Pre-test Loop!
If-Then-Else!
59
Post-test Loop!
Case!
Condensing with respect to the structured programming constructs
Student worksheet !
Triangle Program
Part II"
Unit Testing: White box testing!
!
60
Condensing with respect to the structured programming constructs
Triangle Program
Part II"
Unit Testing: White box testing!
!
61
Condensing with respect to the structured programming constructs (cont.)
Part II"
Unit Testing: White box testing!
!
62
Condensing with respect to the structured programming constructs (cont.)
Part II"
Unit Testing: White box testing!
!
63
Violations of structured programming "
Branching into a loop!
Branching into a decision!
Part II"
Unit Testing: White box testing!
!
64
Branching out of a loop!
Branching out of a decision!
Applicability and Limitation"
§  Control flow testing is the cornerstone of unit testing. It
should be used for all modules of code that cannot be
tested sufficiently through reviews and inspections.
!
§  Its limitation are that the tester must have sufficient
programming skill to understand the code and its
control flow.
!
§  Control flow testing can be very time consuming
because of all modules and basic paths that comprise a
system.!
Part II"
Unit Testing: White box testing!
!
65
66
Ordering the Strategies
All-Paths!
Multiple Condition!
Decision/Condition!
Decision !
Condition!
Statement!
Part II"
Unit Testing: White box testing!
!
White box Testing
Summary"
§  White box testing (Glass box testing, Open box testing, Clear box testing, Structural
testing)!
Ÿ  Control flow testing"
Ÿ  Data flow testing"
Part II"
Unit Testing: White box testing!
!
67
White box Testing, Control flow testing
Summary (cont.)"
Code Coverage (test coverage metrics)!
Levels of Coverage:!
!
§
§
§
§
§
§
§
Statement (Line) coverage!
Decision (Branch) coverage!
Condition coverage!
Multiple Condition coverage!
Decision/Condition coverage!
Loop coverage!
Path coverage!
Part II"
Unit Testing: White box testing!
!
68
Next lecture"
§  White-box testing"
Ÿ  Control flow testing (Chapter 10)!
Ÿ  Data flow testing (Chapter 11)!
Part II"
Unit Testing: White box testing!
!
69
70
Part II"
Unit Testing: White box testing!
!