Business results
Business
modeling process
A Traditional Software
Development Process
Requirements
analysis
System test
Analysis
Integration test
Architectural
design
Unit test
Detailed design
Nov 2002
91.3913
R McFadyen
1
Black Team
Peopleware: Productive Projects and Teams
by Tom Demarco and Timothy Lister
Ch 19: The Black Team
•"legendary" Black Team at IBM in 60’s
•a particularly effective testing team
•the best testers at the company
•testers delighted in finding new ways to break
software
•programmers dreaded having their software go
through Black Team testing
•promoted themselves by dressing all in black
•the spirit of the Black Team has survived
Nov 2002
91.3913
R McFadyen
2
Testing
Unit testing
•White-box testing
•Black-box testing
Nov 2002
91.3913
R McFadyen
3
White-box Testing
•Tests are derived from knowledge of the software’s
construction
•aka structural, glass-box, or clear-box testing
•Tester analyzes the code, the algorithm, to derive tests
Nov 2002
91.3913
R McFadyen
4
Whitebox testing
Tests are derived from knowledge of the software’s construction
Path testing: A whitebox testing technique based on flow of
control
•Construct a flow graph for Cyclomatic Complexity
•gives us the maximum number of tests necessary to cover all
edges in the flow graph
•Design test cases such that each transition is traversed at least
once - examine each condition and select an input for the true
branch and another for the false branch
•Each test should contain an edge that is not contained in any
other test
Nov 2002
91.3913
R McFadyen
5
White-box Testing Example
FindMean(float Mean, FILE ScoreFile)
…see next slide
What is the flow graph for FindMean?
What is the CC?
What tests can we generate from the flowgraph/CC … to
know all paths have been executed?
Nov 2002
91.3913
R McFadyen
6
White-box Testing Example
FindMean(float Mean, FILE ScoreFile)
{ SumOfScores = 0.0; NumberOfScores = 0; Mean = 0;
Read(ScoreFile, Score);
/*Read in and sum the scores*/
while (! EOF(ScoreFile) {
if ( Score > 0.0 ) {
SumOfScores = SumOfScores + Score;
NumberOfScores++;
}
Read(ScoreFile, Score);
}
/* Compute the mean and print the result */
if (NumberOfScores > 0 ) {
Mean = SumOfScores/NumberOfScores;
printf("The mean score is %f \n", Mean);
} else
printf("No scores found in file\n");
}
Nov 2002
91.3913
R McFadyen
7
Prepare for Flow Graph
FindMean (FILE ScoreFile)
{ float SumOfScores = 0.0;
int NumberOfScores = 0;
float Mean=0.0; float Score;
Read(ScoreFile, Score);
2 while (! EOF(ScoreFile) {
3 if (Score > 0.0 ) {
SumOfScores = SumOfScores + Score;
NumberOfScores++;
}
4
5
Read(ScoreFile, Score);
}
/* Compute the mean and print the result */
6 if (NumberOfScores > 0) {
Mean = SumOfScores / NumberOfScores;
printf(“ The mean score is %f\n”, Mean);
} else
printf (“No scores found in file\n”);
9 }
Nov 2002
1
91.3913
R McFadyen
7
8
8
Constructing the Logic Flow Diagram
CC = 11-9+2 = 4
1
Note: If node 4 is included then
node 7 must be included
1, 2, 6, 8, 9 ok
2
1, 2, 6, 7, 9 is not possible
1, 2, 3, 4, 5, 2, 6, 7, 9 ok
3
4
1, 2, 3, 4, 5, 2, 6, 8, 9 is not possible
5
1, 2, 3, 5, 2, 6, 8, 9 ok
6
1, 2, 3, 5, 2, 6, 7, 9 is not possible
7
8
9
Nov 2002
These 3 tests cover all paths!
We didn’t need 4!
Every line of code is tested!
91.3913
R McFadyen
9
Constructing the Logic Flow Diagram
CC = 11-9+2 = 4
1
1, 2, 6, 8, 9
2
Test with no records in
the file
3
4
5
6
7
8
9
Nov 2002
91.3913
R McFadyen
10
Constructing the Logic Flow Diagram
CC = 11-9+2 = 4
1
2
1, 2, 3, 4, 5, 2, 6, 7, 9
3
4
A test with one
positive score
5
6
7
8
9
Nov 2002
91.3913
R McFadyen
11
Constructing the Logic Flow Diagram
CC = 11-9+2 = 4
1
2
1, 2, 3, 5, 2, 6, 8, 9
3
A test with one negative
score
4
5
6
7
8
9
Nov 2002
91.3913
R McFadyen
12
Black-box Testing
• A blackbox test is one that focuses on the input/output
behaviour of a component without considering its
implementation
• The system being tested is a black box, whose behaviour is
determined by studying its inputs and outputs
• aka functional testing since it is concerned with
functionality and not implementation
• Can use equivalence classes and boundary conditions
Nov 2002
91.3913
R McFadyen
13
Equivalence class testing
•
•
•
•
•
A technique to minimize the number of test cases
An equivalence class is a set of tests with common characteristics
A test case is selected for each class
We use our domain knowledge to generate equivalence classes
Example: suppose a method returns the number of days in a
month, given the month and year.
– When we consider months, we could arrive at 3 equivalence
classes:
• months with 31 days
• months with 30 days
• months with 28 or 29 days
– When we consider years, we have two equivalence classes: leap
years and non-leap years
– Combining, we have 6 equivalence classes
Nov 2002
91.3913
R McFadyen
14
Six equivalence classes
Input values
month
year
Equivalence class
31 day month, non-leap year
7
1901
31 day month, leap year
7
1904
30 day month, non-leap year
6
1901
30 day month, leap year
6
1904
28/29 day month, non-leap year
2
1901
28/29 day month, leap year
2
1904
Nov 2002
91.3913
R McFadyen
15
Boundary class testing
•We focus on the boundary conditions of equivalence classes
•Rather than selecting any element in an equivalence class,
boundary testing requires that elements be selected from the
“edges”
Example
•In general, years that are multiples of 4 are leap years; years
that are multiples of 100 are not leap years, unless they are
multiples of 400.
•2000 is a leap year, but 1900 is not
•For Year, 2000 and 1900 are good boundary cases
•0 and 13 would be good boundary cases for month
Nov 2002
91.3913
R McFadyen
16
Ten testing classes
Equivalence class
Input values
month
year
31 day month, non-leap year
7
1901
31 day month, leap year
7
1904
30 day month, non-leap year
6
1901
30 day month, leap year
6
1904
28/29 day month, non-leap year
2
1901
28/29 day month, leap year
2
1904
Leap year divisible by 400
2
2000
Non-leap year divisible by 100
2
1900
Non-positive invalid month
0
1904
positive invalid month
13
1904
Nov 2002
91.3913
R McFadyen
17
Example
How would knowledge of the algorithm (the code) affect your
choices of equivalence classes and boundary classes?
• First, suppose you are testing a search method. What
equivalence and boundary test classes would you use?
What are your black-box test classes?
• Next, suppose you learn the search is a binary search …
how might this knowledge affect your set of test classes
…
Nov 2002
91.3913
R McFadyen
18
Example (continued):
Suppose we have a search module to test. We know:
– the list, elt, must have at least one element
– if the element, key, is found, found will be true and
elt[L] = key
– if the key is not found, found will be false
How do we structure, or organize, our test cases?
Nov 2002
91.3913
R McFadyen
19
Example (continued):
• How do we structure, or organize, our test cases?
• We have two types of searches: successful and
unsuccessful. So, we can partition our search test cases
into two classifications: found and not found
• When it comes to lists of elements, we know that lists of
length 1, are special cases or boundary points. So, we
should have 2 partitions of lists: length 1, and length of
more than one.
• When an element is searched for, it can be found in
boundary positions. So, we can should have 3 partitions:
found at the start, found at the end, and then we should
include where it is found in the middle.
Nov 2002
91.3913
R McFadyen
20
Example (continued):
•combining the partitions
Found
Not Found
1 List of 1 element
2 List of 1 element
List of >1 element
4 In 1st position
3 List of >1 element
5 In last position
6 In middle position
Nov 2002
91.3913
R McFadyen
21
Example (continued):
•sample tests for the 6 partitions
Inputs
List
Key
Outputs
Found, L
1
55
55
true, 1
2
44
55
false, ?
6
55 33 88 44
55
true, 1
4 22 33 6 77 88 44
44
true, 6
5
33 66 77 88 22
77
true, 3
3
22 55 88 66 77
99
false, ?
Nov 2002
91.3913
R McFadyen
22
Example (continued):
•Now, suppose we are examining the algorithm that
searches an ordered list for a specific value in Key.
•On examination we see it’s a binary search and we see
that it really treats the List as having 3 sections:
elements < mid
Nov 2002
midelements > mid
point
91.3913
R McFadyen
23
Example (continued):
•:We can use this knowledge to further refine our
partitioning - we need to test for where the Key we are
looking for is at the boundary points for these partitions.
elements < mid
midelements > mid
point
We’ll add two more tests relating to the boundary points –
finding a key just before, and just after the midpoint.
(Could there be more that we should add related to not
finding a key? What is the control flowgraph?)
Nov 2002
91.3913
R McFadyen
24
Example (continued)
Inputs
List
Key
Outputs
Found, L
55
55
true, 1
false, ?
55
true, 1
4 22 33 44 50 56 61 76
50
true, 4
5 22 33 44 50 56 61 76
76
true, 7
7 22 33 44 50 56 61 76
8 22 33 44 50 56 61 76
3
22 55 88 66 77
44
true, 3
56
99
true, 5
false, ?
1
2
6
Nov 2002
55
44
55 66 77 88 99
91.3913
R McFadyen
25