Software Testing and Quality
Assurance
Lecture 25 – Testing Interactions
(Chapter 6)
1
Lecture Outline


Sampling Test Cases
Orthogonal array testing
2
Sampling test cases


Population: all possible test cases that
can be executed
A sample is a subset of a population
that has been selected based on some
probability distribution.
3
Sampling test cases

Possibilities for determining which test cases
to select

Based on probability distribution


Uniform probability distribution


A probability distribution defines, for each data value in a
population, a set of allowable values and the probability
that value will be selected.
Each value in the population is assigned the same
selection probability
For example, Probability distribution based on the
use profile
4
Sampling test cases (cont...)

A stratified sample is a set of samples
in which each sample represents a
specific subpopulation


Example: select a sample of test cases
from the use cases of each actor
Stratified sample tests are selected from
a series of categories.
5
Sampling test cases (cont...)


Use of random number generator in
sampling
 Advantage: all values have equal
probability
 Disadvantage: test cases cannot be
reproduced (you may let the test driver
record the generated values)
Class family: a set of classes related by
inheritance.
6
Sampling test cases: Orthogonal
array testing (OATS)


Orthogonal arrays provide a specific
sampling technique that seeks to limit
the explosion by defining pair-wise
combinations of a set of interacting
objects.
An orthogonal array is an array of
values in which each column represents
a factor (a variable in an experiment).
7
Sampling test cases: Orthogonal
array testing (OATS)


It represents a specific class family (a
class and its children)
E.g. 3 factors with 3 levels each (27
possibilities), with pair-wise only 9
possibilities
8
Sampling test cases: OATS
Example


Decide how many independent variables will
be tested for interaction. This will map to the
Factors of the array.
Decide the maximum number of values that
each independent variable will take on. This
will map to the Levels of the array.
9
Sampling test cases: OATS
Example



Find a suitable orthogonal array. A
suitable array is one that has at least as
many Factors as needed from Step 1
and has at least as many levels for each
of those factors as decided in Step 2.
Map the Factors and values onto the
array.
Transcribe the Runs into test cases.
10
Sampling test cases: OATS
example

Number of states:






Class A: 2
Class B: 3
Class P: 1
Class C: 2
Class D: 3
Class E: 3
* State transition diagrams are not shown
11
Sampling test cases: OATS
example (cont...)

Step 1: Identify all factors: sending
hierarchy, receiving hierarchy,
parameter position in the message (six
factors in the example: class A
hierarchy, class P hierarchy, class C
hierarchy and factors associated with
each class hierarchy)
12
Sampling test cases: OATS
example (cont...)

Step 2: Determine levels for each factor
by considering the set of possible
values:



One factor has one level: the parameter
class family only has one member: P
Two factors have a maximum of two levels;
B and C.
Three factors have a maximum of three
levels: A, D and E.
13
Sampling test cases: OATS
example (cont...)



Step 3: Locate a
standard
orthogonal array
that fits the
problem.
6 factors
3 levels
14
Sampling test cases: OATS
example (cont...)
15
Sampling test cases: OATS
example (cont...)

Step 4: Establish a mapping from each
factor onto the integers in the array so that
standard array can be interpreted


The first column in L18 can be used to represent
the sender class family which has two classes: A
and B (1 corresponds to A class and 2
corresponds to class B)
The second column in L18 when there is a
difference in the number of levels (class A has 2
states and class B has 3 states).
16
Sampling test cases: OATS
example (cont...)




The third column in L18 represents the
parameter hierarchy that only has one class,
P. (any value in the third column represents
P).
The fourth column represents the state of P
(which there are two).
The fifth column represents the class C
hierarchy, which has three members.
The sixth column represents the states of the
C, D, and E classes.
17
Sampling test cases: OATS
example (cont...)

Step 5: Construct test cases based on
the mapping and the rows in the table.

Each row in the orthogonal array specifies
one specific test case.
18
Sampling test cases: OATS
example— result interpretation

The 10th
column
represents:


An instance of
B in state 1
Is to send a
message by
passing an
instance of
class P in state
19
Sampling test cases: OATS another
example

Consider a web page with three distinct
sections (Top, Middle, and Bottom) that
can be individually shown or hidden by
the user. You wish to test the
interactions of the different sections.
20
Sampling test cases: OATS another
example (cont...)




There are three independent variables
(the sections of the page).
Each variable can take on two values
(hidden or visible).
An L4 orthogonal array will do the job —
two levels for the values and three
factors for the variables.
Mapping the values onto the array
where Hidden=0 and Visible=1
21
Sampling test cases: OATS another
example (cont...)
OA before mapping factors
Factor 1
Factor 2
Factor 3
Run 1
0
0
0
Run 2
0
1
1
Run 3
1
0
1
Run 4
1
1
0
OA after mapping factors
Top
Middle
Bottom
Test case 1
Hidden
Hidden
Hidden
Test case 2
Hidden
Visible
Visible
Test case 3
Visible
Hidden
Visible 22
Sampling test cases: adequacy
criteria for OATS



Exhaustive: all possible combinations
of all factors are considered.
Minimal: only the interactions between
the base classes from each hierarchy
are tested.
Random: the tester haphazardly selects
cases from several of the classes.
23
Sampling test cases: adequacy
criteria for OATS


Representative: a uniform sample that
ensures that every class is tested to
some level.
Weighted representative: Add cases
to the representative approach based
on relative importance or risk
associated with the class.
24
Key points




Possibilities for determining which test cases to
select: based on probability distribution, uniform
probability distribution and probability distribution
based on the use profile.
A stratified sample is a set of samples in which each
sample represents a specific subpopulation
An orthogonal array is an array of values in which
each column represents a factor (a variable in an
experiment).
Adequacy criteria for OATS: exhaustive, minimal,
random, representative, and weighted representative.
25