2/2/09
COMP 3705
Advanced Software
Engineering:
Software Testing
Prof. Matt Rutherford
Class 04 - 2009-02-02
(c) 2009 Matthew J. Rutherford
For Next Week
  Readings
  Sections 3.3, 3.4, 3.5
  Homework Problems
  Section 3.2, Exercise #1, predicates (1), (5), (9) and (10)
  Section 3.3, Exercise #1
Class 04 - 2009-02-02
(c) 2009 Matthew J. Rutherford
1
2/2/09
Looking Ahead
  Next class (Class 5)
  Finish logic coverage criteria
  February 16 (Class 6)
  MIDTERM (first 90 minutes)
  Input Space Partitioning
  February 23 (Class 7)
  Syntax Based Testing
Class 04 - 2009-02-02
(c) 2009 Matthew J. Rutherford
Today
  Logic Criteria
  Basics and Definitions
  Application to source code
Class 04 - 2009-02-02
(c) 2009 Matthew J. Rutherford
2
2/2/09
Ch. 3 : Logic Coverage
Applied to
Applied
to
Applied
to
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
5
Covering Logic Expressions
(3.1)
•  Logic expressions show up in many situations
•  Covering logic expressions is required by the US Federal
Aviation Administration for safety critical software
•  Logical expressions can come from many sources
–  Decisions in programs
–  FSMs and statecharts
–  Requirements
•  Tests are intended to choose some subset of the total number of
truth assignments to the expressions
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
6
3
2/2/09
Logic Predicates and Clauses
•  A predicate is an expression that evaluates to a boolean value
•  Predicates can contain
–  boolean variables
–  non-boolean variables that contain >, <, ==, >=, <=, !=
–  boolean function calls
•  Internal structure is created by logical operators
–  ¬ – the negation operator
–  ∧ – the and operator
–  ∨ – the or operator
–  → – the implication operator
–  ⊕ – the exclusive or operator
–  ↔ – the equivalence operator
•  A clause is a predicate with no logical operators
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
7
Examples
•  (a < b) ∨ f (z) ∧ D ∧ (m >= n*o)
•  Four clauses:
–  (a < b) – relational expression
–  f (z) – boolean-valued function
–  D – boolean variable
–  (m >= n*o) – relational expression
•  Most predicates have few clauses
–  It would be nice to quantify that claim!
•  Sources of predicates
–  Decisions in programs
–  Guards in finite state machines
–  Decisions in UML activity graphs
–  Requirements, both formal and informal
–  SQL queries
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
8
4
2/2/09
Translating from English
•  “I am interested in SWE 637 and CS 652”
•  course = swe637 OR course = cs652
Humans have trouble
translating from English
to Logic
•  “If you leave before 6:30 AM, take Braddock to 495, if you leave
after 7:00 AM, take Prosperity to 50, then 50 to 495”
•  time < 6:30 → path = Braddock ∨ time > 7:00 → path = Prosperity
•  Hmm … this is incomplete !
•  time < 6:30 → path = Braddock ∨ time >= 6:30 → path = Prosperity
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
9
© Ammann & Offutt
Section 3.1 Problems
  Exercise #2: Write the predicate (only the predicate) to
represent the requirement: “List all the wireless mice
that either retail for more than $100 or for which the
store has more than 20 items. Also list non-wireless
mice that retail for more than $50.”
Class 04 - 2009-02-02
(c) 2009 Matthew J. Rutherford
5
2/2/09
Testing and Covering Predicates
(3.2)
•  We use predicates in testing as follows :
–  Developing a model of the software as one or more predicates
–  Requiring tests to satisfy some combination of clauses
•  Abbreviations:
–  P is the set of predicates
–  p is a single predicate in P
–  C is the set of clauses in P
–  Cp is the set of clauses in predicate p
–  c is a single clause in C
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
11
Predicate and Clause Coverage
•  The first (and simplest) two criteria require that each predicate
and each clause be evaluated to both true and false
•  When predicates come from conditions on edges, this is equivalent
to edge coverage
•  PC does not evaluate all the clauses, so …
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
12
6
2/2/09
Predicate Coverage Example
((a < b) ∨ D) ∧ (m >= n*o)
predicate coverage
Predicate = true
a = 5, b = 10, D = true, m = 1, n = 1, o = 1
= (5 < 10) ∨ true ∧ (1 >= 1*1)
= true ∨ true ∧ TRUE
= true
Predicate = false
a = 10, b = 5, D = false, m = 1, n = 1, o = 1
= (10 < 5) ∨ false ∧ (1 >= 1*1)
= false ∨ false ∧ TRUE
= false
13
© Ammann & Offutt
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
Clause Coverage Example
((a < b) ∨ D) ∧ (m >= n*o)
Clause coverage
(a < b) = true
a = 5, b = 10
D = true
(a < b) = false
D = true
a = 10, b = 5
m >= n*o = true
m >= n*o = false
m = 1, n = 1, o = 1
m = 1, n = 2, o = 2
D = false
D = false
false cases
Two tests
true cases
1) a = 5, b = 10, D = true, m = 1, n = 1, o = 1
2) a = 10, b = 5, D = false, m = 1, n = 2, o = 2
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
14
7
2/2/09
Problems with PC and CC
•  PC does not fully exercise all the clauses, especially in the
presence of short circuit evaluation
•  CC does not always ensure PC
–  That is, we can satisfy CC without causing the predicate to be both true
and false
–  This is definitely not what we want !
•  The simplest solution is to test all combinations …
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
15
Combinatorial Coverage
•  CoC requires every possible combination
•  Sometimes called Multiple Condition Coverage
a<b D
m >= n*o
1 T
T
T
2 T
T
F
3 T
F
T
4 T
F
F
5 F
T
T
6 F
T
F
7 F
F
T
8 F
F
F
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
((a < b) ∨ D) ∧ (m >= n*o)
T
F
T
F
T
F
F
F
© Ammann & Offutt
16
8
2/2/09
Combinatorial Coverage
•  This is simple, neat, clean, and comprehensive …
•  But quite expensive!
•  2N tests, where N is the number of clauses
–  Impractical for predicates with more than 3 or 4 clauses
•  The literature has lots of suggestions – some confusing
•  The general idea is simple:
Test each clause independently from the other clauses
•  Getting the details right is hard
•  What exactly does “independently” mean ?
•  The book presents this idea as “making clauses active” …
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
17
Active Clauses
•  Clause coverage has a weakness : The values do not always
make a difference
•  Consider the first test for clause coverage, which caused each
clause to be true:
–  (5 < 10) ∨ true ∧ (1 >= 1*1)
•  Only the first clause counts !
•  To really test the results of a clause, the clause should be the
determining factor in the value of the predicate
Determination :
A clause ci in predicate p, called the major clause,
determines p if and only if the values of the
remaining minor clauses cj are such that changing
ci changes the value of p
•  This is considered to make the clause active
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
18
9
2/2/09
Determining Predicates
P=A∨B
P=A∧B
if B = true, p is always true.
if B = false, p is always false.
so if B = false, A determines p.
so if B = true, A determines p.
if A = false, B determines p.
if A = true, B determines p.
•  Goal : Find tests for each clause when the clause determines the
value of the predicate
•  This is formalized in several criteria that have subtle, but very
important, differences
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
19
Active Clause Coverage
p=a∨b
1)
a = true, b = false
2)
a = false, b = false
3)
a = false, b = true
4)
a = false, b = false
a is major clause
b is major clause
Duplicate
•  This is a form of MCDC, which is required by the FAA for
safety critical software
•  Ambiguity : Do the minor clauses have to have the same values
when the major clause is true and false?
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
20
10
2/2/09
Resolving the Ambiguity
p = a ∨ (b ∧ c)
Major clause : a
a = true, b = false, c = true
Is this allowed ?
a = false, b = false, c = false
c = false
•  This question caused confusion among testers for years
•  Considering this carefully leads to three separate criteria :
–  Minor clauses do not need to be the same
–  Minor clauses do need to be the same
–  Minor clauses force the predicate to become both true and false
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
21
General Active Clause Coverage
•  This is complicated !
•  It is possible to satisfy GACC without satisfying predicate
coverage
•  We really want to cause predicates to be both true and false !
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
22
11
2/2/09
Restricted Active Clause Coverage
•  This has been a common interpretation by aviation developers
•  RACC often leads to infeasible test requirements
•  There is no logical reason for such a restriction
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
23
Correlated Active Clause Coverage
•  A more recent interpretation
•  Implicitly allows minor clauses to have different values
•  Explicitly satisfies (subsumes) predicate coverage
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
24
12
2/2/09
CACC and RACC
a
b
c
a ∧ (b ∨ c)
1 T
T
T
T
1
2
T
F
T
5
F
T
T
2
a
3
T
TT
TT
5
FF
T
T
F
6
6
FF
T
F
F
3
7
FF
F
T
F
7
major clause
CACC can be satisfied by choosing any
of rows 1, 2, 3 AND any of rows 5, 6, 7 –
a total of nine pairs
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
a
b
c
a ∧ (b ∨ c)
T
T
T
T
T
T
F
T
T
F
F
T
F
T
T
F
F
T
T
F
F
F
T
T
F
T
F
a
T
F
F
major clause
RACC can only be satisfied by one of the
three pairs above
25
© Ammann & Offutt
CACC versus RACC
  Textbook Page 110-111
  Dependency between variables often prohibits some of
the truth-table values
Class 04 - 2009-02-02
(c) 2009 Matthew J. Rutherford
13
2/2/09
Inactive Clause Coverage
•  The active clause coverage criteria ensure that “major” clauses
do affect the predicates
•  Inactive clause coverage takes the opposite approach – major
clauses do not affect the predicates
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
27
General and Restricted ICC
•  Unlike ACC, the notion of correlation is not relevant
–  ci does not determine p, so cannot correlate with p
•  Predicate coverage is always guaranteed
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
28
14
2/2/09
Logic Coverage Criteria Subsumption
Combinatorial
Clause Coverage
COC
Restricted Active
Clause Coverage
RACC
Restricted Inactive
Clause Coverage
RICC
Correlated Active
Clause Coverage
CACC
General Inactive
Clause Coverage
GICC
General Active
Clause Coverage
GACC
Clause
Coverage
CC
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
Predicate
Coverage
PC
© Ammann & Offutt
29
Making Clauses Determine a Predicate
•  Finding values for minor clauses cj is easy for simple predicates
•  But how to find values for more complicated predicates ?
•  Definitional approach:
–  pc=true is predicate p with every occurrence of c replaced by true
–  pc=false is predicate p with every occurrence of c replaced by false
•  To find values for the minor clauses, connect pc=true and
•
pc=false with exclusive OR
pc = pc=true ⊕ pc=false
After solving, pc describes exactly the values needed for c to
determine p
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
30
15
2/2/09
Examples
p=a∨b
pa = pa=true ⊕ pa=false
= (true ∨ b) XOR (false ∨ b)
= true XOR b
=¬b
p=a∧b
pa = pa=true ⊕ pa=false
= (true ∧ b) ⊕ (false ∧ b)
= b ⊕ false
=b
p = a ∨ (b ∧ c)
pa = pa=true ⊕ pa=false
= (true ∨ (b ∧ c)) ⊕ (false ∨ (b ∧ c))
= true ⊕ (b ∧ c)
= ¬ (b ∧ c)
=¬ b∨¬c
•  “NOT b ∨ NOT c” means either b or c can be false
•  RACC requires the same choice for both values of a, CACC does not
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
31
A More Subtle Example
p = ( a ∧ b ) ∨ ( a ∧ ¬ b)
pa = pa=true ⊕ pa=false
= ((true ∧ b) ∨ (true ∧ ¬ b)) ⊕ ((false ∧ b) ∨ (false ∧ ¬ b))
= (b ∨ ¬ b) ⊕ false
= true ⊕ false
= true
p = ( a ∧ b ) ∨ ( a ∧ ¬ b)
pb = pb=true ⊕ pb=false
= ((a ∧ true) ∨ (a ∧ ¬ true)) ⊕ ((a ∧ false) ∨ (a ∧ ¬ false))
= (a ∨ false) ⊕ (false ∨ a)
=a⊕a
= false
•  a always determines the value of this predicate
•  b never determines the value – b is irrelevant !
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
32
16
2/2/09
Infeasible Test Requirements
•  Consider the predicate:
(a > b ∧ b > c) ∨ c > a
•  (a > b) = true, (b > c) = true, (c > a) = true is infeasible
•  As with graph-based criteria, infeasible test requirements have to
be recognized and ignored
•  Recognizing infeasible test requirements is hard, and in general,
undecidable
•  Software testing is inexact – engineering, not science
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
33
Logic Coverage Summary
•  Predicates are often very simple
–  PC may be enough
–  CoC is practical
•  Control software often has many complicated predicates, with
lots of clauses
•  Question … why don’t complexity metrics count the number of
clauses in predicates?
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
34
17
2/2/09
Finding Satisfying Values
  The criteria help us determine the boolean value that
each clause must evaluate to
  To apply this, we need to pick actual values to plug in
that will result in the desired boolean value for each
clause
  There is no requirement that values used be consistent
even though the criteria limit choices in terms of the
clause values
Class 04 - 2009-02-02
(c) 2009 Matthew J. Rutherford
Section 3.2 Problem
  Exercise #1, predicate (2)
  p = a OR (b AND c)
Class 04 - 2009-02-02
(c) 2009 Matthew J. Rutherford
18
2/2/09
Logic Expressions from Source
Predicates are derived from decision statements in programs
  In programs, most predicates have less than four clauses

–  Wise programmers actively strive to keep predicates simple
When a predicate only has one clause, COC, ACC, ICC, and CC
all collapse to predicate coverage (PC)
  Applying logic criteria to program source is hard because of
reachability and controllability:

–  Reachability : Before applying the criteria on a predicate at a particular statement,
we have to get to that statement
–  Controllability : We have to find input values that indirectly assign values to the
variables in the predicates
–  Variables in the predicates that are not inputs to the program are called internal
variables

These issues are illustrated through an example in the following
slides …
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
37
TriTyp Program
  Textbook pages 121-122
  Types:
  Equilateral – all sides the same length
  Scalene – all sides have different lengths
  Isosceles – two sides have the same length
  Triangle inequality:
  The sum of the lengths of any two sides always exceeds the
length of the third side
Class 04 - 2009-02-02
(c) 2009 Matthew J. Rutherford
19
2/2/09
Triang
(pg 1 of 5)
1
2
3
4
5
6
7
8
// Jeff Offutt -- Java version Feb 2003
// The old standby: classify triangles
// Figures 3.2 and 3.3 in the book.
import java.io.*;
class trityp
{
private static String[] triTypes = { "", // Ignore 0.
"scalene", "isosceles", "equilateral", "not a valid
triangle"};
9 private static String instructions = "This is the ancient
TriTyp program.\nEnter three integers that represent the lengths
of the sides of a triangle.\nThe triangle will be categorized as
either scalene, isosceles, equilateral\nor invalid.\n";
10
11 public static void main (String[] argv)
12 { // Driver program for trityp
13 int A, B, C;
14 int T;
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
Triang
© Ammann & Offutt
39
(pg 2 of 5)
16 System.out.println (instructions);
17 System.out.println ("Enter side 1: ");
18 A = getN();
19 System.out.println ("Enter side 2: ");
20 B = getN();
21 System.out.println ("Enter side 3: ");
22 C = getN();
23 T = Triang (A, B, C);
24
25 System.out.println ("Result is: " + triTypes [T]);
26 }
27
28 // ====================================
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
40
20
2/2/09
Triang
(pg 3 of 5)
29 // The main triangle classification method
30 private static int Triang (int Side1, int Side2, int Side3)
31 {
32 int tri_out;
33
34 // tri_out is output from the routine:
35 // Triang = 1 if triangle is scalene
36 // Triang = 2 if triangle is isosceles
37 // Triang = 3 if triangle is equilateral
38 // Triang = 4 if not a triangle
39
40 // After a quick confirmation that it’s a legal
41 // triangle, detect any sides of equal length
42 if (Side1 <= 0 || Side2 <= 0 || Side3 <= 0)
43 {
44
tri_out = 4;
45
return (tri_out);
46 }
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
Triang
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
© Ammann & Offutt
41
(pg 4 of 5)
tri_out = 0;
if (Side1 == Side2)
tri_out = tri_out + 1;
if (Side1 == Side3)
tri_out = tri_out + 2;
if (Side2 == Side3)
tri_out = tri_out + 3;
if (tri_out == 0)
{ // Confirm it’s a legal triangle before declaring
// it to be scalene
if (Side1+Side2 <= Side3 || Side2+Side3 <= Side1 ||
Side1+Side3 <= Side2)
tri_out = 4;
else
tri_out = 1;
return (tri_out);
}
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
42
21
2/2/09
Triang
(pg 5 of 5)
67 /* Confirm it’s a legal triangle before declaring */
68 /* it to be isosceles or equilateral */
69
70 if (tri_out > 3)
71
tri_out = 3;
72 else if (tri_out == 1 && Side1+Side2 > Side3)
73
tri_out = 2;
74 else if (tri_out == 2 && Side1+Side3 > Side2)
75
tri_out = 2;
76 else if (tri_out == 3 && Side2+Side3 > Side1)
77
tri_out = 2;
78 else
79
tri_out = 4;
80 return (tri_out);
81 } // end Triang
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
43
Ten Triang Predicates
42: (Side1 <= 0 || Side2 <= 0 || Side3 <= 0)
49: (Side1 == Side2)
51: (Side1 == Side3)
53: (Side2 == Side3)
55: (triOut == 0)
59: (Side1+Side2 <= Side3 || Side2+Side3 <= Side1 ||
Side1+Side3 <= Side2)
70: (triOut > 3)
72: (triOut == 1 && Side1+Side2 > Side3)
74: (triOut == 2 && Side1+Side3 > Side2)
76: (triOut == 3 && Side2+Side3 > Side1)
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
44
22
2/2/09
Reachability for Triang Predicates
42: True
49: P1 = s1>0 && s2>0 && s3>0
51: P1
53: P1
Need to solve for the
55: P1
internal variable triOut
59: P1 && triOut = 0
62: P1 && triOut = 0
&& (s1+s2 > s3) && (s2+s3 > s1) && (s1+s3 > s2)
70: P1 && triOut != 0
72: P1 && triOut != 0 && triOut <= 3
74: P1 && triOut != 0 && triOut <= 3 && (triOut !=1 || s1+s2<=s3)
76: P1 && triOut != 0 && triOut <= 3 && (triOut !=1 || s1+s2<=s3)
&& (triOut !=2 || s1+s3<=s2)
78: P1 && triOut != 0 && triOut <= 3 && (triOut !=1 || s1+s2<=s3)
&& (triOut !=2 || s1+s3 <= s2) && (triOut !=3 || s2+s3 <= s1)
© Ammann & Offutt
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
45
Solving for Internal Variable triOut
At line 55, triOut has a value in the range (0 .. 6)
triOut = 0
1
2
3
4
5
6
s1!=s2
s1=s2
s1!=s2
s1!=s2
s1=s2
s1!=s2
s1=s2
&&
&&
&&
&&
&&
&&
&&
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
s1!=s3
s1!=s3
s1=s3
s1!=s3
s1!=s3
s1=s3
s1=s3
&&
&&
&&
&&
&&
&&
&&
s2!=s3
s2!=s3
s2!=s3
s2=s3
s2=s3
s2=s3
s2=s3
© Ammann & Offutt
Contradiction
Contradiction
46
23
2/2/09
Reachability for Triang Predicates
(solved for triOut – reduced)
42: True
49: P1 = s1>0 && s2>0 && s3>0
Looks complicated, but a
51: P1
lot of redundancy
53: P1
55: P1
59: P1 && s1 != s2 && s2 != s3 && s2 != s3
(triOut = 0)
62: P1 && s1 != s2 && s2 != s3 && s2 != s3
(triOut = 0)
&& (s1+s2 > s3) && (s2+s3 > s1) && (s1+s3 > s2)
70: P1 && P2 = (s1=s2 || s1=s3 || s2=s3)
(triOut != 0)
72: P1 && P2 && P3 = (s1!=s2 || s1!=s3 || s2!=s3)
(triOut <= 3)
74: P1 && P2 && P3 && (s1 != s2 || s1+s2<=s3)
76: P1 && P2 && P3 && (s1 != s2 || s1+s2<=s3)
&& (s1 != s3 || s1+s3<=s2)
78: P1 && P2 && P3 && (s1 != s2 || s1+s2<=s3)
&& (s1 != s3 || s1+s3<=s2) && (s2 != s3 || s2+s3<=s1)
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
47
Predicate Coverage
These values are
“don’t care”, needed
to complete the test.
p42: (S1 <= 0 || S2 <= 0 || S3 <= 0)
p49: (S1 == S2)
p51: (S1 == S3)
p53: (S2 == S3)
p55: (triOut == 0)
p59: (S1+S2 <= S3 ||
S2+S3 <= S1 ||
S1+S3 <= S2)
p70: (triOut > 3)
p72: (triOut == 1 && S1+S2 > S3)
p74: (triOut == 2 && S1+S3 > S2)
p76: (triOut == 3 && S2+S3 > S1)
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
A
0
1
1
1
1
T
B
0
1
1
1
2
C EO
0 4
1 3
1 3
1 3
3 4
“EO” – expected
output
F
A B C EO
1 1 1 3
1 2 2 2
1 2 2 2
2 1 2 2
1 1 1 3
1 2 3 4
2 3 4 1
1
2
2
3
2
2
2
4
1
2
3
2
1
3
2
2
© Ammann & Offutt
3
2
2
2
2
2
4
2
3
4
2
2
2
4
4
4
48
24
2/2/09
Clause Coverage
p42: (S1 <= 0)
(S2 <= 0 )
(S3 <= 0)
p59: (S1+S2 <= S3 )
(S2+S3 <= S1)
(S1+S3 <= S2)
p72: (triOut == 1)
(S1+S2 > S3)
p74: (triOut == 2)
(S1+S3 > S2)
p76: (triOut == 3)
(S2+S3 > S1)
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
A
0
1
1
2
6
2
2
2
T
F
B C EO A B C EO
1 1 4 1 1 1 3
0 1 4 1 1 1 3
1 0 4 1 1 1 3
3 6 4 2 3 4 1
2 3 4 2 3 4 1
6 3 4 2 3 4 1
2 3 2 2 3 2 2
2 3 2 2 2 5 4
2
2
3
3
3
3
2
2
2
2
2
2
2
2
2
2
3
2
1
5
2
5
2
2
2
2
1
2
2
4
4
4
© Ammann & Offutt
49
Correlated Active Clause Coverage
p42: (S1 <= 0 || S2 <= 0 || S3 <= 0)
p59: (S1+S2 <= S3 || S2+S3 <= S1 ||
S1+S3 <= S2)
p72: (triOut == 1 && S1+S2 > S3)
s1=s2 && s1!=s3 && s2!=s3
p74: (triOut == 2 && S1+S3 > S2)
s1!=s2 && s1=s3 && s2!=s3
p76: (triOut == 3 && S2+S3 > S1)
s1!=s2 && s1!=s3 && s2=s3
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
T
F
f
f
T
F
f
f
T
F
t
T
F
t
T
F
t
f
f
T
f
f
f
T
f
t
t
F
t
t
F
t
t
F
f
f
f
T
f
f
f
T
A
0
1
1
1
2
2
6
2
2
2
2
2
2
2
3
1
5
© Ammann & Offutt
B C EO
1 1
4
1 1
3
0 1
4
1 0
4
3 6
4
3 4
1
2 3
4
6 3
4
2 3
2
3 3
2
2 5
4
3 2
2
3 3
2
5 2
4
2 2
2
2 2
4
2 2
4
At least one
pair of sides
must be
equal.
50
25
2/2/09
Program Transformation Issues
if ((a && b) || c) {
S1;
}
else {
S2;
}
Transform (1)?
Transform (2)?
d = a && b;
e = d || c;
if (e) {
S1;
}
else {
S2;
}
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
if (a) {
if (b)
S1;
else {
if (c) {
S1;
else {
S2;
}
}
else {
if (c) {
S1;
else {
S2;
}
© Ammann & Offutt
51
Problems with Transformed Programs

Maintenance is certainly harder
with Transform (1)
–  Not recommended!

Coverage on Transform (1)

CACC
PC
T
X
T F T
T
X
X
T F F
F
X
X
–  Structure used by logic criteria is
F T T
“lost”
–  Hence CACC on transform 2 only
F T F
requires 3 tests
–  Note: Mutation analysis (Chapter
F F T
5) addresses this problem
T
Coverage on Transform (2)
Bottom Line: Logic coverage
criteria are there to help you!
F F F
F
CACC(2)
X
T T F
–  PC on transform does not imply
CACC on original
–  CACC on original does not imply
PC on transform

a b c (a∧b)∨c
T T T
T
X
X
X
X
X
T
F
X
26
2/2/09
Summary : Logic Coverage for Source Code

Predicates appear in decision statements
–  if, while, for, etc.

Most predicates have less than four clauses
–  But some applications have predicates with many clauses
The hard part of applying logic criteria to source is resolving the
internal variables
  Non-local variables (class, global, etc.) are also input variables if
they are used
  If an input variable is changed within a method, it is treated as
an internal variable thereafter
  To maximize effect of logic coverage criteria:

–  Avoid transformations that hide predicate structure
Introduction to Software Testing (Ch 3), www.introsoftwaretesting.com
© Ammann & Offutt
53
Section 3.3 Problems
  Exercise #2
Class 04 - 2009-02-02
(c) 2009 Matthew J. Rutherford
27