Testing OO Programs
(7.1, 2.2.2, 2.4.2)
Course Software Testing & Verification
2014/15
Wishnu Prasetya
Plan
• The goal is to discuss how to test OO
programs; to that end we also need some
concepts from:
• Data flow perspective to testing
• Integration testing
2
Basic idea
P(x,y) { if (y<x) y = x+1 ;
return x*y }
• The two paths of P above will carry the same value of x
“defined” at the beginning to the return statement.
• A plain control flow graph does not describe how
“data” flows through the program; so we can’t infer
which execution paths need to be tested (or not) to
test the influence of a certain variable on the program
3
Data-flow based approach
(sec. 2.2.2)
P1(x,y) { if (y<x) y = x+1 ;
return x*y }
n0
def (n0) = { x,y }
n1
use(n1) = {x,y}
(then)
use{n2) = {x}
def(n2) = {y}
n2
n3
use(n3) = {x,y}
• Data flow graph : CFG, where the nodes and edges are
decorated with information about the variables used and
defined by each node/edge.
• In my examples I avoid decorating on the edges.
• Some subtlety on the granularity of the nodes/edges  later.
4
Terminologies
use = {x,y}
def ={x}
use = {y}
def = {y}
n4
n2
n1
n3
use = {x,y}
def = { x,y }
use = {x,y}
def = 
n5
use = {x,y}
def = 
du-paths wrt x:
13, 134, 135
12, 1234, 1235,
45
Not du-path wrt x:
1345 (not def-clear)
• Def/use graph: see above. In general, def/use labels are
allowed on the edges as well.
• A path p = [i,...,k] is def-clear wrt v iff v  def(j) and v 
def(e) for all nodes j and edges e in p between i and k.
• It (def-clear) is a du-path wrt v iff vdef(i) and vuse(k), and
furthermore it is a simple path.
5
Block’s granularity
x = x+1
n
use(n) = {x}
def (n) = { x }
x=0
y = x+1 ;
n
use(n) = {x}
def (n) = { x,y }
x=0
y = x+1
n1
n2
use(n1) = {}
def (n1) = { x}
use(n2) = {x}
def (n2) = { y}
• When a node both defines and uses the same variable x, we
will assume that it uses x first, then assigns to it, as in
x = ... x ...
• If this assumption would be broken (middle example)  split
the node. Else this will conflict the intention of your definition
of du-path.
• Check 2.3.2 on how to map a source code to a def/use graph. 6
Data-flow based coverage
(Rapps-Weyuker 1985, Frankl-Weyuker 1988)
use = {x}
def ={x}
n1
use = {x}
def = { x}
n2
use = {x}
def =
n3
n4
use ={x}
def = 
• Def-path set du(i,v): the set of all du-paths wrt v, that starts at i.
• Def-pair set du(i,j,v): the set of all du-paths wrt v, that starts at i,
and ends at j.
• Same concept of touring.
• (C 2.9, All Defs Covrg, ADC) For each def-path set S = du(i,v), TR
includes at least one member of S.
7
Data-flow based coverage
use = {x}
def ={x}
n1
use = {x}
def = { x }
n2
use = {x}
def =
n3
n4
ADC TR:
use ={x} 12,
def = 
234
AUC TR:
12
23 , 232,
234
ADUC TR:
12
23 , 232,
234, 24
• (C 2.10, All Uses Covrg, AUC) For each def-pair set S =
du(i,j,v), TR includes at least one member of S.
• (C2.11, All du-paths Covrg, ADUPC) For each def-pair
set S = du(i,j,v), TR includes all members of S.
• Note: the above example only has one variable,
namely x; consequently all TRs above only concern x.
8
Overview of subsumption relations
complete path covrg
ADUPC
AUC
ADC
prime path covrg
edge-pair covrg
edge covrg
node covrg
• PPC  ADUPC, because every simple path is a subpath of some
prime path.
• AUC  EC .... under the following assumptions:
– every use (of v) is preceded by a def.
– every def reaches at least one use.
– multiple branches e.g. as in if(g) then S1 else S2 use the same variables (if
S1 uses x, then S2 at some point also uses x, vice versa).
9
Summary
• We learned another set of white-box test-coverage
criteria, namely data-flow based coverage
• an alternative to prime-path-based testing, if the
latter becomes too expensive.
10
Integration Test
• Integration test: to test if a composition of program
units works correctly.
• “Hidden” in Section 2.4 about using design elements
(e.g. your UML models) as your coverage criterion.
2.4.1 says: “testing software based on design elements
is usually associated with integration testing”.
• 2.4.2 is actually more about integration testing rather
than design-element-based testing.
11
Data flow approach to Integration Test
(2.4.2)
A
f (a) { ... g(e) ...
}
g(x) { ... }
B
• Imagine method f from module A calls method/API g
from module B.
• Idea 1: construct the combined CFG of both, then use
prime path coverage criteria. This blows up.
• Idea 2: focus on the flow of the “coupling data”
around the call point.
12
Some terminologies
(1) g(x)
(1) a = ... ;b = ...
(2) a>b
(4) b = g(b)
(2) x<=a
(3) a = ...
(4) return ...
(3) x==a
(5) return ...
• caller, callee, actual parameters, formal parameters
• parameter coupling, shared variable coupling,
external device coupling
• More general concept: coupling variable  a
‘variable’ defined in one ‘unit’, and used in the other.
13
Coupling du-path
(Jin-Offutt, 1998)
(1) g(x)
(1) a = ... ;b = ...
(2) a>b
(4) b = g(b)
(2) x<=a
(3) a = ...
(4) return ...
(3) x==a
(5) return ...
• (Def 2.40) Last-def of v : set of nodes defining v, from which
there is a def-clear path (wrt v) through the call-site, to a use in
the other unit.
• Can be at the caller or callee! Example: f1, f3, g4, g5.
• (Def 2.41) First-use (of v) E.g.: g2, f4
• Coupling du-path of v is a path from v’s last-def to its first-use.
14
First use at the callee ..
• When you just pass a variable as in g(b), it makes
sense to define g’s first use to be the first time g uses
the parameter (location g.3 in the previous example).
• But when you pass a complex expression like g(a+b)
 what are g’s first uses of a and b?
 in this case it make sense to define g’s first uses
(of a and b) to be at the call location itself.
15
Integration Test-Requirement
(1) a = ... ;b = ...
(2) a>b
(4) b = g(b)
(1) g(x)
(3) a = ...
(2) x<=a
(4) return ...
(3) x==a
(5) return ...
Coupling vars:
a, b, g.return
Coupling du-paths :
a: [f1,f2,g1,g2]
[f3,f4,g1,g2]
b: [f1,f2,g1,g2]
[f1,f2,f3,g1,g2]
g.return: [g4,f4]
[g5,f4]
• ADC  All Coupling Def Cov. : for every last-def d of each
coupling var v, TR includes one coupling du-path starting from d.
• AUC  All Coupling Use Cov. : for every last-def d and first-use u
of v, TR includes one couple du-path from d to u.
16
OO is powerful, but also comes new
sources of errors
• A function behaves “cleanly”; a procedure does not.
• An OO program is more complicated:
– side effects on instance variables and static
variables
– access control (priv, pub, default, protected, ...)
– dynamic binding
– interaction through inheritance
17
Anomalies
• Simply mean non-standard situations. Not
necessarily errors, but they tend to be error prone.
AO list 9 anomalies related to inheritance and
dynamic binding:
– Inconsistent Type Use (ITU)
– State definition anomaly (SDA) , ... 7 more
• AO also implicitly discuss two more: deep yoyo and
data-flow anomaly.
18
Inconsistent Type Use (ITU) anomaly
List
+insertElemAt(i:int)
+remElemAt(i:int)
Stack
+push(x)
+pop()
This is a quite common subclassing architecture that people use. However note that this
does not prevent a Stack to be used as a List, which is error prone (it allows you to call
insertElemAt or remElemAt on a Stack).
19
State Definition Anomaly (SDA)
UnitItem
scale = 1
price() { return 1/scale }
ScalableItem
setScale(s) { scale = s }
• ScalableItem (subclass) directly manipulates the part of
its state inheritted from UnitItem (rather than through
UnitItem’s method). This error prone: may break
UnitItem’s class invariant.
20
State definition inconsistency due to
state variable hiding (SDIH)
Item
price
tax
netPrice() { return price * (1 + tax) }
setPrice(p) { price = p }
OnSaleItem
price
sale(p) { price = 0.8 p }
• The shadowing of price done by OnSaleItem is error
prone; notice the method netPrice (that it also
inherits!) would use the old price.
21
State Visibility Anomaly (SVA)
A
-x
+ incr()
B
C
+ incr2()
C’s needs a method incr2 that would increase x by 2. However it
can’t get to x, because it is private to A. It can however calls
super.incr() twice. So far it is ok. Now imagine someone else
changes B by overriding incr. This suddenly changes the behavior
of C; as it now calls B’s incr instead.
22
Anomalous Construction Behavior 1 (ACB1)
Item
price
tax
Item() { reset() ; tax=0 }
reset() { price = 1 }
ExpensiveItem
baseprice
ExpensiveItem() { super() ; baseprice=10 }
reset() { price = baseprice }
• Inheritence may radically change the orginal
behavior.
• Item’s constructor calls reset().
• ExpensiveItem’s constructor calls Item’s constructor,
but implicitly forces it to call the new reset() !
• Called a Yoyo-effect phenomenon.
23
Call graph
f
g
h
Here, we’ll define f  g means f contains a call to g in its
body (note that it does not mean that in the runtime f
will actually call g; it may depend on its parameters) The
two branches from f in the example says nothing about
the order in which the children will be called by f.
• Here, it is a graph describing which program “units” call
which. .
24
The call graph of d() in ABC
yoyo graph, Alexander-Offut 2000
A
+d () { g() }
+g () { h() }
+h () { i() }
+i () { j() }
+j () {}
A
A
B
+h () { i() ...}
+ i() { super.i() ...}
+k() { }
g()
h()
i()
j()
d()
g()
h()
i()
j()
h()
i()
h()
i()
h()
i()
B
A
B
C
+j () { k() }
d()
C
d()
g()
j()
k()
j()
25
Data flow anomaly
A
+d ()
+g ()
+h ()
+i ()
+j ()
def x
A
g()
h()
i()
def x
A
B
+h ()
+i ()
d()
use x
B
d()
g()
j()
use x
h()
i()
h()
i()
j()
Not necessarily an error. E.g. if at the runtime the use of x in j() when it is called
from B’s i() is actually unfeasible. But this is still error prone (anomaly)
26
Check the remaining anomalies
yourself
•
•
•
•
State Defined Incorrectly (SDI)
Indirect Inconsistent State Definition (IISD)
Anomalous Construction Behavior 2 (ACB2)
Incomplete Construction (IC)
27
Testing your OO programs
•
•
•
•
Intra-method testing (method-level unit-testing)
Intra-class testing, as done by class-level unit testing.
Inter-class testing.
The first two levels are usually not enough to expose
errors related to access control, interaction through
inheritance, and dynamic binding.
• Let’s now take a look at inter-class testing.
28
Inter-class testing
class E {
f(x) { A o = FactoryA(x ) ;
...
o.m() }
}
• Testing the interaction of class A and E : treat this as
an integration testing problem  we already have a
solution (data-flow based integration test), but
additionally we need to quantify over the subclasses
of A, due to dynamic binding (the exact m called in f
depends on the type of actual type of o ).
29
Testing more complicated interaction
class E {
f(x) { A o = FactoryA(x ) ;
o.m() ;
...
o.n()
}
// (1)
// (2)
// (3)
• The way f(x) interacts with o is more complicated. There
may be various combinations of which m and n are
actually called, leading to complex overall behavior.
• Let’s extend our idea of integration test, to also cover
such interaction.
30
Terminologies
f(x) { (1) A o = FactoryA(x ) ;
(2) o.m() ;
...
(3) o.n()
}
• f itself is here called the coupling method.
• The field o.x is “indirectly” defined (i-def) if the field is
updated. Analogously, use and i-use.
• Focus on coupling within f due to variables o.v (indirectly) def.
by o.m(), and used by o.n().  antecedent and consequent.
31
Terminologies
f(x) { (1) A o = FactoryA(x ) ;
(2) o.m() ;
...
(3) o.n()
}
• Coupling sequence C: a pair of antecedent m, and
consequence n, such that there is at least 1x def-clear path
from m to n with respect to some field o.x defined in m and
used in n.
• o.x is called a coupling var (of C)
• The set of all coupling vars of C  coupling set.
32
With polymorphism the situation will now
look like this...
Notice how different types of the context o may give different coupling variables
as well as coupling paths.
33
Binding triple
• Each coupling sequence induces one or more binding
“triples”; each specifying the coupling variables for
various types of the context-var o. For example, this
could be the binding triples of sequence
o.m() o.n()
A
B
C
type of o
coupling vars
A
{ o.v }
B
{ o.u }
C
{ o.u , o.v }
binding triples of c = o.m() o.n()
34
Inter-class coverage test criteria
A
B
C
type(o)
coupling vars
coupling paths
A
{ o.v }
o.v : { p1 }
B
{ o.u }
o.u : { q1, q2 }
C
{ o.u , o.v }
o.u : { r1 } , o.v :{r2}
binding triples of c =
o.m() o.n() ,
extended with infos
over coupling paths.
• (Def 7.53) All-Coupling-Sequences (ACS): for every
coupling sequence c in the coupling method f , TR
includes at least one coupling path of c.  ignore
polymorphism.
• (Def 7.54) All-Poly-Classes (APC): for every binding
triple t of c, TR includes at least one coupling path of
t.  ignore that c may have multiple coupling vars. 35
Inter-class coverage test criteria
A
B
C
type(o)
coupling vars
coupling paths
A
{ o.v }
o.v : { p1 }
B
{ o.u }
o.u : { q1, q2 }
C
{ o.u }
o.u : { r1 }
• (Def 7.55) All-Coupling-Defs-Uses (ACDU): for every
coupling var v of c, TR includes at least one of v’s
coupling path.
• (Def 7.56) All-Poly-Coupling-Defs-Uses (APCDU): for
every coupling var v of every binding triple of c, TR
includes at least one of v’s coupling path.
36