Introduction to Software
Testing
Chapter 8.2
Syntactic Logic Coverage
Criteria
(Disjunctive Normal Form)
Paul Ammann & Jeff Offutt
http://www.cs.gmu.edu/~offutt/softwaretest/
Disjunctive Normal Form
• Common Representation for Boolean Functions
– Slightly Different Notation for Operators
– Slightly Different Terminology
• Basics:
– A literal is a clause or the negation (overstrike) of a clause
• Examples: a, a
– A term is a set of literals connected by logical “and”
• “and” is denoted by adjacency instead of 
• Examples: ab, ab, ab for a  b, a  ¬ b, ¬ a  ¬ b
– A (disjunctive normal form) predicate is a set of terms connected
by “or”
• “or” is denoted by + instead of 
• Examples: abc + ab + ac
• Terms are also called “implicants”
– If a term is true, that implies the predicate is true
Introduction to Software Testing, Edition 2 (Ch 8)
© Ammann & Offutt
2
Implicant Coverage
(8.2.1)
• Obvious coverage idea : Make each implicant evaluate to
“true”
– Problem : Only tests “true” cases for the predicate
– Solution : Include DNF representations for negation
Implicant Coverage (IC) : Given DNF representations of
a predicate f and its negation f, for each implicant in f
and f,TR contains the requirement that the implicant
evaluate to true.
• Example: f = ab + bc
f = b + ac
• Implicants: { ab, bc, b, ac }
• Possible test set: {TTF, FFT}
• Observation: IC is relatively weak
Introduction to Software Testing, Edition 2 (Ch 8)
© Ammann & Offutt
3
Improving on Implicant Coverage
(8.2.2)
• Additional Definitions :
– A proper subterm is a term with one or more clauses removed
• Example: abc has 6 proper subterms: a, b, c, ab, ac, bc
– A prime implicant is an implicant such that no proper subterm is
also an implicant
• Example: f = ab + abc
• Implicant ab is a prime implicant
• Implicant abc is not a prime implicant (due to proper subterm ac)
– A redundant implicant is an implicant that can be removed
without changing the value of the predicate
• Example: f = ab + ac + bc
• ab is redundant
• Predicate can be written: ac + bc
Introduction to Software Testing, Edition 2 (Ch 8)
© Ammann & Offutt
4
Unique True Points
• A minimal DNF representation is one with only prime, non-
redundant implicants
• A unique true point with respect to a given implicant is an
assignment of truth values so that
– The given implicant is true, and
– All other implicants are false
• A unique true point test focuses on just one implicant
• A minimal representation guarantees the existence of at
least one unique true point for each implicant
Multiple Unique True Point Coverage (MUTP) : Given
minimal DNF representations of a predicate f, for each
implicant i, choose unique true points (UTPs) such that clauses
not in i take on values T and F.
Introduction to Software Testing, Edition 2 (Ch 8)
© Ammann & Offutt
5
Unique True Point Example
• Consider again : f = ab + bc
– Implicants : { ab, bc, b, ac }
– Each of these implicants is prime
– None of these implicants is redundant
• Unique true points :
–
–
–
–
ab: {TTT}
bc: {FTF}
b: {FFF, TFF, TFT}
ac: {FTT}
• Note: One possible (minimal) test set satisfying MUTP
– For third clause, need first and third tests, but not second
– Note that MUTP is infeasible for the other three clauses
• not enough UTPs for clauses not in implicant to take on all truth values
Introduction to Software Testing, Edition 2 (Ch 8)
© Ammann & Offutt
6
Near False Points
(8.2.3)
• A near false point with respect to a clause c in implicant i is
an assignment of truth values such that f is false, but if c is
negated (and all other clauses left as is), i (and hence f)
evaluates to true
• Relation to determination: at a near false point, c determines
fUnique True Point and Near False Point Pair Coverage
– Hence we should
expect
relationship
ACC criteria of a
(CUTPNFP)
: Given
a minimal
DNFtorepresentation
predicate f, for each clause c in each implicant i,TR
contains a unique true point for i and a near false point for
c such that the points differ only in the truth value of c.
• Note that definition only mentions f, and not f
• Clearly, CUTPNFP subsumes RACC
Introduction to Software Testing, Edition 2 (Ch 8)
© Ammann & Offutt
7
CUTPNFP Example
• Consider f = ab + cd
– Implicant ab has 3 unique true points : {TTFF, TTFT, TTTF}
• For clause a, we can pair unique true point TTFF with near false point FTFF
• For clause b, we can pair unique true point TTFF with near false point TFFF
– Implicant cd has 3 unique true points : {FFTT, FTTT, TFTT}
• For clause c, we can pair unique true point FFTT with near false point FFFT
• For clause d, we can pair unique true point FFTT with near false point FFTF
• CUTPNFP set : {TTFF, FFTT, TFFF, FTFF, FFTF, FFFT}
– First two tests are unique true points; others are near false points
• Rough number of tests required: # implicants * # literals
Introduction to Software Testing, Edition 2 (Ch 8)
© Ammann & Offutt
8
The MNFP and MUMCUT Criteria
(8.2.3)
The next two criteria provide enough scaffolding to make
guarantees about fault detection (see later slides)
Multiple Near False Point Coverage (MNFP) : Given a
minimal DNF representation of a predicate f, for each
literal c in each implicant i,TR choose near false points
(NFPs) such that clauses not in i take on values T and F.
MUMCUT : Given a minimal DNF representation of a
predicate f, apply MUTP, CUTPNFP, and MNFP.
Introduction to Software Testing, Edition 2 (Ch 8)
© Ammann & Offutt
9
DNF Fault Classes
• ENF: Expression Negation Fault f = ab+c
•
•
•
•
•
•
•
•
f’ = ab+c
TNF: Term Negation Fault
f = ab+c
f’ = ab+c
TOF: Term Omission Fault
f = ab+c
f’ = ab
LNF: Literal Negation Fault
f = ab+c
f’ = ab+c
LRF: Literal Reference Fault
f = ab + bcd f’ = ad + bcd
LOF: Literal Omission Fault
f = ab + c
f’ = a + c
LIF: Literal Insertion Fault
f = ab + c
f’ = ab + bc
ORF+: Operator Reference Fault f = ab + c
f’ = abc
ORF*: Operator Reference Fault f = ab + c
f’ = a + b + c
Key idea is that fault classes are related with respect to testing :
Test sets guaranteed to detect certain faults are also
guaranteed to detect additional faults
Introduction to Software Testing, Edition 2 (Ch 8)
© Ammann & Offutt
10
Fault Detection Relationships
Literal Insertion
Fault
LIF
Term Omission
Fault
TOF
Operator
Reference Fault
ORF+
Literal
Reference Fault
LRF
Literal Negation
Fault
LNF
Term Negation
Fault
TNF
Literal Omission
Fault
LOF
Operator
Reference Fault
ORF*
Expression
Negation Fault
ENF
Introduction to Software Testing, Edition 2 (Ch 8)
© Ammann & Offutt
11
•
Karnaugh Maps for Testing Logic
Expressions
(8.2.4)
Fair Warning
– We use, rather than teach, Karnaugh Maps
– Newcomers to Karnaugh Maps probably need a tutorial
• Suggestion: Google “Karnaugh Map Tutorial”
• Our goal: Apply Karnaugh Maps to concepts used to test
logic expressions
– Identify when a clause determines a predicate
– Identify the negation of a predicate
– Identify prime implicants and redundant implicants
– Identify unique true points
– Identify unique true point / near false point pairs
• No new material here on testing
– Just fast shortcuts for concepts already presented
Introduction to Software Testing, Edition 2 (Ch 8)
© Ammann & Offutt
12
K-Map:
A Clause Determines a Predicate
• Consider the predicate : f = b + ac + ac
• Suppose we want to identify when b determines f
• The dashed line highlights where b changes value
– If two cells joined by the dashed line have different values for f,
then b determines f for those two cells
– b determines f: ac + ac (but NOT at ac or ac )
• Repeat for clauses a and c
ab
c
0
00
01
11
t
t
t
t
t
1
Introduction to Software Testing, Edition 2 (Ch 8)
10
t
© Ammann & Offutt
13
K-Map: Negation of a predicate
• Consider the predicate: f = ab + bc
• Draw the Karnaugh Map for the negation
– Identify groups
– Write down negation: f = b + a c
ab
c
00
01
0
1
t
Introduction to Software Testing, Edition 2 (Ch 8)
11
10
ab
c
00
01
t
t
0
t
t
1
t
© Ammann & Offutt
11
10
t
t
14
K-Map:
Prime and Redundant Implicants
• Consider the predicate: f = abc + abd + abcd + abcd + acd
• Draw the Karnaugh Map
• Implicants that are not prime: abd, abcd, abcd, acd
• Redundant implicant: abd
• Prime implicants
– Three: ad, bcd, abc
– The last is redundant
– Minimal DNF representation
• f = ad + bcd
ab
cd
00
00
11
10
t
t
01
11
10
Introduction to Software Testing, Edition 2 (Ch 8)
01
© Ammann & Offutt
t
t
t
t
15
K-Map: Unique True Points
• Consider the predicate: f = ab + cd
• Three unique true points for ab
– TTFF, TTFT, TTTF
– TTTT is a true point, but not a unique true point
• Three unique true points for cd
ab
– FFTT, FTTT, TFTT
• Unique true points for f
f = ac + bc + ad + bd
– FTFT,TFFT, FTTF, TFTF
00
11
10
cd
00
t
01
t
11
t
10
Introduction to Software Testing, Edition 2 (Ch 8)
01
© Ammann & Offutt
t
t
t
t
16
MUTP: Multiple Unique True Points
• For each implicant find unique true points (UTPs) so that
– Literals not in implicant take on values T and F
• Consider the DNF predicate: ab
cd
– f = ab + cd
• For implicant ab
– Choose TTFT, TTTF
• For implicant cd
– Choose FTTT, TFTT
• MUTP test set
– {TTFT, TTTF, FTTT, TFTT}
Introduction to Software Testing, Edition 2 (Ch 8)
00
01
11
00
t
01
t
11
t
10
© Ammann & Offutt
t
t
10
t
t
17
CUTPNFP: Corresponding Unique True
Point Near False Point Pairs
• Consider the DNF predicate: f = ab + cd
• For implicant ab
– For a, choose UTP, NFP pair
• TTFF, FTFF
ab
cd
– For b, choose UTP, NFP pair
• TTFT, TFFT
• For implicant cd
– For c, choose UTP, NFP pair
– For d, choose UTP, NFP pair
• FFTT, FFTF
Introduction to Software Testing, Edition 2 (Ch 8)
11
t
01
t
t
10
• Possible CUTPNFP test set
– {TTFF, TTFT, FFTT
//UTPs
FTFF, TFFT, FFFT, FFTF} //NFPs
01
00
11
• FFTT, FFFT
00
© Ammann & Offutt
t
t
10
t
t
18
MNFP : Multiple Near False Points
• Find NFP tests for each literal such that all literals not in
the term attain F and T
• Consider the DNF predicate:
– f = ab + cd
• For implicant ab
– Choose FTFT, FTTF for a
– Choose TFFT, TFTF for b
ab
cd
00
01
11
00
t
• For implicant cd
– Choose FTFT, TFFT for c
– Choose FTTF, TFTF for d
01
t
• MNFP test set
– {TFTF, TFFT, FTTF, TFTF}
10
11
t
t
t
10
t
t
• Example is small, but generally MNFP is large
Introduction to Software Testing, Edition 2 (Ch 8)
© Ammann & Offutt
19
Minimal-MUMCUT Criterion
Kaminski/Ammann (ICST 2009)
• Minimal-MUMCUT uses low level criterion feasibility
analysis
– Adds CUTPNFP and MNFP only when necessary
• Minimal-MUMCUT guarantees detecting LIF, LRF, LOF
– And thus all 9 faults in the hierarchy
For
Each
Term
MUTP
feasible?
Test Set =
MUTP + NFP
Introduction to Software Testing, Edition 2 (Ch 8)
For
Each
Literal
In
Term
CUTPNFP
feasible?
MNFP
Test Set =
MUTP + MNFP
Test Set =
MUTP + CUTPNFP
© Ammann & Offutt
20