Domain testing
Tor Stålhane
Domain testing revisited
We have earlier looked at domain testing as
a simple strategy for selecting test cases.
We will now take this a step further.
Testers have frequently observed that
domain boundaries are particularly fault
prone and should therefore be carefully
checked.
Predicates
We will assume that all predicates are
simple. This implies that they contain only
one relational operator. Allowable
operators are =, ><, >=, <=, < and >.
Predicates that are not simple can be split
into two or more simple predicates.
Path conditions
Each path thorough a system is defined by a
path condition – the conjunction of all
predicates encountered along this path.
An input traverses a path if and only if the
path condition is fulfilled.
The path condition defines a path domain,
which is the set of all inputs that cause
the path to be executed.
Path domains – 1
A path domain is surrounded by a boundary,
consisting of one or more borders where
each border corresponds to a predicate.
A border is
• Closed if defined by a predicate containing
=, =< or >=. Closed borders belong to the
path domain.
• Open if defined by a predicate containing
><, > or >. Open borders do not belong to
the path domain.
Path domains – 2
Note the difference between open and
closed domains.
Closed: what is
on the line belongs
to the domain
Open: what is off the line
– below or above –
belongs to the domain
X
X
X
Domain error
A domain error occurs if an input traverses
the wrong path through the program.
We have no way to know the correct borders
and there is no unique correct version of
the program.
When domain error occurs along a path, it
may be thought of as being caused by one
of the given borders being different from
the correct one.
Path domains
P1
S1
S2
P1 and P2: {S1, S3,S4}
P1 and not P2: {S1, S3, S5}
S3
not P1 and P2: {S2, S3, S4}
P2
not P1 and not P2: {S2, S3, S5}
S4
S5
ON and OFF points – 1
The test strategy is a strategy for selecting
ON and OFF points, defined as follows:
• ON point for a
– Closed border lies on the border
– Open border lies close to the border and
satisfies the inequality relation
• OFF point lies close to the border and on
the open side or – alternatively – does not
satisfy the path condition associated with
this border
ON and OFF points – 2
The ON and OFF points are used as follows:
• For testing a closed border we use
– Two ON points to identify the border
– One OFF point, to test that the correct border
does not lie on the open side of the border
• For testing an open border the role of the
ON and OFF points are reversed.
The strategy can be extended to Ndimensional space by using N ON points
ON and OFF points – 3
If the border line has V vertices, we will need
• One ON point close to each vertex.
• One OFF point per vertex at a uniform
distance from the border.
In all cases, it is important that the OFF
points are as close as possible to the ON
points
Example – two-dimensional space
Correct border
OFF
ON
ON
Given border
Example for an open border: ><, < or >. Border outside the line
Example – two-dimensional space
Correct border
OFF
ON
ON
Given border
OFF
Example for an closed border: “=“ predicate. Border on the line
The problem of size
The main problem with this strategy is the
cost. Let us assume we have 20 input
variables. Then the suggested strategy
would need:
• For each > or <: 20 ON points and one
OFF point
• For each = or ><: 20 ON points plus two or
three OFF points
Ten predicates would require 210 – 230 test
cases.
The problem of precision
The strategy described require the ON
points to lie exactly on the border.
For any pair of real numbers, there is always
a third real number that lies between them.
For a computer, however, this is not the
case, due to limited precision.
Thus, there exist borders for which no ON
point can be represented in the computer.
A simplified strategy
We will drop the requirement that the border
can be exactly identified. Then we can
also drop the requirement that the ON
point lies exactly on the border. This
remove the precision problem.
In addition we can reduce the number of
points by one per border. The only error
that will not be detected is if the real
border passes between an ON and an
OFF point. Thus, these two points need to
be close.
Simplified use of On and OFF points
OFF
OFF
=, ><
>, <
ON
ON
OFF
Effectiveness
OFF
d
>, <
Assume that
• m1 is the smallest real value
• m2 is the largest real value
ON
The length of L, called M, is
M = m2 – m1 +1
L
P(not detect) = d / M
Code containing array references
Code segment. The array b has 10 elements.
i = x + 3;
IF b[i] <=5 THEN ,,,
ELSE …
We need three predicates:
• b[x + 3] <= 5 – the original predicate
• x + 3 >= 1 – not below lower bound
• x + 3 >= 10 – not above upper bound
Non-linear borders – 1
Everything discussed till now has been
based on the assumption that the real
border is linear. If this is not true, the
domain strategy might fail.
In the example on the next slide, the ON
points are on the border, OFF point is off
the border but nevertheless the given
border is wrong.
Non-linear borders – 2
Correct border
OFF
ON
ON
Given border
A simple algorithm
We can apply domain testing as follows:
1. Select a test case and run it. This will
cause one path to be traversed.
2. Identify the borders of this path and test
them with ON and OFF points
3. If the OFF point belongs to a new path
then this path is selected for testing
otherwise check another OFF point.
4. Terminate when no new path can be
found
Simple algorithm - example
A
1
2
B
C
3
IF x in A THEN S1
ELSE
IF x in B THEN S2
ELSE
S3
When to use domain testing
Domain testing, as it is described here,
requires that we know how the input
partition the input space into input
domains.
Thus, it is only possible to use it for small
chunks of code.
Acknowledgement
This set of slides is based on the paper
“A Simplified Domain-Testing Strategy”
by B. Jeng and E.J. Weyuker.