advertisement

Introduction to Software Testing Chapter 8.2 Syntactic Logic Coverage Criteria (Disjunctive Normal Form) Paul Ammann & Jeff Offutt http://www.cs.gmu.edu/~offutt/softwaretest/ Disjunctive Normal Form • Common Representation for Boolean Functions – Slightly Different Notation for Operators – Slightly Different Terminology • Basics: – A literal is a clause or the negation (overstrike) of a clause • Examples: a, a – A term is a set of literals connected by logical “and” • “and” is denoted by adjacency instead of • Examples: ab, ab, ab for a b, a ¬ b, ¬ a ¬ b – A (disjunctive normal form) predicate is a set of terms connected by “or” • “or” is denoted by + instead of • Examples: abc + ab + ac • Terms are also called “implicants” – If a term is true, that implies the predicate is true Introduction to Software Testing, Edition 2 (Ch 8) © Ammann & Offutt 2 Implicant Coverage (8.2.1) • Obvious coverage idea : Make each implicant evaluate to “true” – Problem : Only tests “true” cases for the predicate – Solution : Include DNF representations for negation Implicant Coverage (IC) : Given DNF representations of a predicate f and its negation f, for each implicant in f and f,TR contains the requirement that the implicant evaluate to true. • Example: f = ab + bc f = b + ac • Implicants: { ab, bc, b, ac } • Possible test set: {TTF, FFT} • Observation: IC is relatively weak Introduction to Software Testing, Edition 2 (Ch 8) © Ammann & Offutt 3 Improving on Implicant Coverage (8.2.2) • Additional Definitions : – A proper subterm is a term with one or more clauses removed • Example: abc has 6 proper subterms: a, b, c, ab, ac, bc – A prime implicant is an implicant such that no proper subterm is also an implicant • Example: f = ab + abc • Implicant ab is a prime implicant • Implicant abc is not a prime implicant (due to proper subterm ac) – A redundant implicant is an implicant that can be removed without changing the value of the predicate • Example: f = ab + ac + bc • ab is redundant • Predicate can be written: ac + bc Introduction to Software Testing, Edition 2 (Ch 8) © Ammann & Offutt 4 Unique True Points • A minimal DNF representation is one with only prime, non- redundant implicants • A unique true point with respect to a given implicant is an assignment of truth values so that – The given implicant is true, and – All other implicants are false • A unique true point test focuses on just one implicant • A minimal representation guarantees the existence of at least one unique true point for each implicant Multiple Unique True Point Coverage (MUTP) : Given minimal DNF representations of a predicate f, for each implicant i, choose unique true points (UTPs) such that clauses not in i take on values T and F. Introduction to Software Testing, Edition 2 (Ch 8) © Ammann & Offutt 5 Unique True Point Example • Consider again : f = ab + bc – Implicants : { ab, bc, b, ac } – Each of these implicants is prime – None of these implicants is redundant • Unique true points : – – – – ab: {TTT} bc: {FTF} b: {FFF, TFF, TFT} ac: {FTT} • Note: One possible (minimal) test set satisfying MUTP – For third clause, need first and third tests, but not second – Note that MUTP is infeasible for the other three clauses • not enough UTPs for clauses not in implicant to take on all truth values Introduction to Software Testing, Edition 2 (Ch 8) © Ammann & Offutt 6 Near False Points (8.2.3) • A near false point with respect to a clause c in implicant i is an assignment of truth values such that f is false, but if c is negated (and all other clauses left as is), i (and hence f) evaluates to true • Relation to determination: at a near false point, c determines fUnique True Point and Near False Point Pair Coverage – Hence we should expect relationship ACC criteria of a (CUTPNFP) : Given a minimal DNFtorepresentation predicate f, for each clause c in each implicant i,TR contains a unique true point for i and a near false point for c such that the points differ only in the truth value of c. • Note that definition only mentions f, and not f • Clearly, CUTPNFP subsumes RACC Introduction to Software Testing, Edition 2 (Ch 8) © Ammann & Offutt 7 CUTPNFP Example • Consider f = ab + cd – Implicant ab has 3 unique true points : {TTFF, TTFT, TTTF} • For clause a, we can pair unique true point TTFF with near false point FTFF • For clause b, we can pair unique true point TTFF with near false point TFFF – Implicant cd has 3 unique true points : {FFTT, FTTT, TFTT} • For clause c, we can pair unique true point FFTT with near false point FFFT • For clause d, we can pair unique true point FFTT with near false point FFTF • CUTPNFP set : {TTFF, FFTT, TFFF, FTFF, FFTF, FFFT} – First two tests are unique true points; others are near false points • Rough number of tests required: # implicants * # literals Introduction to Software Testing, Edition 2 (Ch 8) © Ammann & Offutt 8 The MNFP and MUMCUT Criteria (8.2.3) The next two criteria provide enough scaffolding to make guarantees about fault detection (see later slides) Multiple Near False Point Coverage (MNFP) : Given a minimal DNF representation of a predicate f, for each literal c in each implicant i,TR choose near false points (NFPs) such that clauses not in i take on values T and F. MUMCUT : Given a minimal DNF representation of a predicate f, apply MUTP, CUTPNFP, and MNFP. Introduction to Software Testing, Edition 2 (Ch 8) © Ammann & Offutt 9 DNF Fault Classes • ENF: Expression Negation Fault f = ab+c • • • • • • • • f’ = ab+c TNF: Term Negation Fault f = ab+c f’ = ab+c TOF: Term Omission Fault f = ab+c f’ = ab LNF: Literal Negation Fault f = ab+c f’ = ab+c LRF: Literal Reference Fault f = ab + bcd f’ = ad + bcd LOF: Literal Omission Fault f = ab + c f’ = a + c LIF: Literal Insertion Fault f = ab + c f’ = ab + bc ORF+: Operator Reference Fault f = ab + c f’ = abc ORF*: Operator Reference Fault f = ab + c f’ = a + b + c Key idea is that fault classes are related with respect to testing : Test sets guaranteed to detect certain faults are also guaranteed to detect additional faults Introduction to Software Testing, Edition 2 (Ch 8) © Ammann & Offutt 10 Fault Detection Relationships Literal Insertion Fault LIF Term Omission Fault TOF Operator Reference Fault ORF+ Literal Reference Fault LRF Literal Negation Fault LNF Term Negation Fault TNF Literal Omission Fault LOF Operator Reference Fault ORF* Expression Negation Fault ENF Introduction to Software Testing, Edition 2 (Ch 8) © Ammann & Offutt 11 • Karnaugh Maps for Testing Logic Expressions (8.2.4) Fair Warning – We use, rather than teach, Karnaugh Maps – Newcomers to Karnaugh Maps probably need a tutorial • Suggestion: Google “Karnaugh Map Tutorial” • Our goal: Apply Karnaugh Maps to concepts used to test logic expressions – Identify when a clause determines a predicate – Identify the negation of a predicate – Identify prime implicants and redundant implicants – Identify unique true points – Identify unique true point / near false point pairs • No new material here on testing – Just fast shortcuts for concepts already presented Introduction to Software Testing, Edition 2 (Ch 8) © Ammann & Offutt 12 K-Map: A Clause Determines a Predicate • Consider the predicate : f = b + ac + ac • Suppose we want to identify when b determines f • The dashed line highlights where b changes value – If two cells joined by the dashed line have different values for f, then b determines f for those two cells – b determines f: ac + ac (but NOT at ac or ac ) • Repeat for clauses a and c ab c 0 00 01 11 t t t t t 1 Introduction to Software Testing, Edition 2 (Ch 8) 10 t © Ammann & Offutt 13 K-Map: Negation of a predicate • Consider the predicate: f = ab + bc • Draw the Karnaugh Map for the negation – Identify groups – Write down negation: f = b + a c ab c 00 01 0 1 t Introduction to Software Testing, Edition 2 (Ch 8) 11 10 ab c 00 01 t t 0 t t 1 t © Ammann & Offutt 11 10 t t 14 K-Map: Prime and Redundant Implicants • Consider the predicate: f = abc + abd + abcd + abcd + acd • Draw the Karnaugh Map • Implicants that are not prime: abd, abcd, abcd, acd • Redundant implicant: abd • Prime implicants – Three: ad, bcd, abc – The last is redundant – Minimal DNF representation • f = ad + bcd ab cd 00 00 11 10 t t 01 11 10 Introduction to Software Testing, Edition 2 (Ch 8) 01 © Ammann & Offutt t t t t 15 K-Map: Unique True Points • Consider the predicate: f = ab + cd • Three unique true points for ab – TTFF, TTFT, TTTF – TTTT is a true point, but not a unique true point • Three unique true points for cd ab – FFTT, FTTT, TFTT • Unique true points for f f = ac + bc + ad + bd – FTFT,TFFT, FTTF, TFTF 00 11 10 cd 00 t 01 t 11 t 10 Introduction to Software Testing, Edition 2 (Ch 8) 01 © Ammann & Offutt t t t t 16 MUTP: Multiple Unique True Points • For each implicant find unique true points (UTPs) so that – Literals not in implicant take on values T and F • Consider the DNF predicate: ab cd – f = ab + cd • For implicant ab – Choose TTFT, TTTF • For implicant cd – Choose FTTT, TFTT • MUTP test set – {TTFT, TTTF, FTTT, TFTT} Introduction to Software Testing, Edition 2 (Ch 8) 00 01 11 00 t 01 t 11 t 10 © Ammann & Offutt t t 10 t t 17 CUTPNFP: Corresponding Unique True Point Near False Point Pairs • Consider the DNF predicate: f = ab + cd • For implicant ab – For a, choose UTP, NFP pair • TTFF, FTFF ab cd – For b, choose UTP, NFP pair • TTFT, TFFT • For implicant cd – For c, choose UTP, NFP pair – For d, choose UTP, NFP pair • FFTT, FFTF Introduction to Software Testing, Edition 2 (Ch 8) 11 t 01 t t 10 • Possible CUTPNFP test set – {TTFF, TTFT, FFTT //UTPs FTFF, TFFT, FFFT, FFTF} //NFPs 01 00 11 • FFTT, FFFT 00 © Ammann & Offutt t t 10 t t 18 MNFP : Multiple Near False Points • Find NFP tests for each literal such that all literals not in the term attain F and T • Consider the DNF predicate: – f = ab + cd • For implicant ab – Choose FTFT, FTTF for a – Choose TFFT, TFTF for b ab cd 00 01 11 00 t • For implicant cd – Choose FTFT, TFFT for c – Choose FTTF, TFTF for d 01 t • MNFP test set – {TFTF, TFFT, FTTF, TFTF} 10 11 t t t 10 t t • Example is small, but generally MNFP is large Introduction to Software Testing, Edition 2 (Ch 8) © Ammann & Offutt 19 Minimal-MUMCUT Criterion Kaminski/Ammann (ICST 2009) • Minimal-MUMCUT uses low level criterion feasibility analysis – Adds CUTPNFP and MNFP only when necessary • Minimal-MUMCUT guarantees detecting LIF, LRF, LOF – And thus all 9 faults in the hierarchy For Each Term MUTP feasible? Test Set = MUTP + NFP Introduction to Software Testing, Edition 2 (Ch 8) For Each Literal In Term CUTPNFP feasible? MNFP Test Set = MUTP + MNFP Test Set = MUTP + CUTPNFP © Ammann & Offutt 20