The UML Testing Profile
- Tutorial at the ECOOP 2004 www.fokus.fraunhofer.de/u2tp
Ina Schieferdecker, Technical University Berlin/FOKUS
Øystein Haugen, University of Oslo
June 2004
adapted to OMS-seminar May 2005
© U2TP Consortium
Agenda
z Motivation
z Basics
of Testing
z Overview
z Example
of the UML Testing Profile
Test Specifications
z The
UML Profile and the MOF Model
z The
Mappings
z Concluding
ECOOP, Oslo, June 2004
Remarks
2
© U2TP Consortium
Agenda
¾ Motivation
z Basics
of Testing
z Overview
z Example
of the UML Testing Profile
Test Specifications
z The
UML Profile and the MOF Model
z The
Mappings
z Concluding
ECOOP, Oslo, June 2004
Remarks
3
© U2TP Consortium
Testing Today
z
Is
z
ƒ Important
ƒ Means to obtain approval
ƒ Time critical
z
But often
ƒ
ƒ
ƒ
ƒ
ƒ
Rarely practiced
Unsystematic
Performed by hand
Error-prone
Considered being
destructive
ƒ Uncool
„If you are a bad
programmer you might be
a tester“
Conjecture:
There is a lack of appropriate test methods and techniques
ECOOP, Oslo, June 2004
4
© U2TP Consortium
Testing is …
z
A technical process
z
Performed by experimenting with a system
z
In a controlled environment following a specified
procedure
z
With the intent of observing one or more
characteristics of the system
z
By demonstrating the deviation of the system’s actual
status from the required status/specification.
ECOOP, Oslo, June 2004
5
© U2TP Consortium
Goals of the UML Testing Profile
z
Definition of a testing profile to capture all information that would
be needed by different test processes
ƒ To allow black-box testing (i.e. at UML interfaces) of computational
models in UML
z
A testing profile based upon UML 2.0
ƒ That enables the test definition and test generation based on
structural (static) and behavioral (dynamic) aspects of UML models,
and
ƒ That is capable of inter-operation with existing test technologies for
black-box testing
z
Define
ƒ Test purposes for computational UML models, which should be
related to relevant system interfaces
ƒ Test components, test configurations and test system interfaces
ƒ Test cases in an implementation independent manner
ECOOP, Oslo, June 2004
6
© U2TP Consortium
U2TP Partners
zA
consortium of testers, UML vendors and users
dedicated to make UML applicable for software testing
z
z
Submitters
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ
ƒ iLogix
ƒ ScapaTechnologies
ƒ IRISA
Ericsson
IBM
FOKUS
Motorola
Rational
Softeam
Telelogic
University of Lübeck
ECOOP, Oslo, June 2004
Supporters
7
© U2TP Consortium
Agenda
z Motivation
¾ Basics
of Testing
z Overview
z Example
of the UML Testing Profile
Test Specifications
z The
UML Profile and the MOF Model
z The
Mappings
z Concluding
ECOOP, Oslo, June 2004
Remarks
8
© U2TP Consortium
Types of Testing
Level
system
system
integration
unit
functionality
functionality
load/performance
load/performance
Accessibility
box white box
black box grey box
black
robustness
interoperability
interoperability
usability
…
Aspect
ECOOP, Oslo, June 2004
9
© U2TP Consortium
Test Concepts: Black-Box Testing
Test Case
Stimulus
Response
• Assignment
of a
Test Verdict
Port
System Under Test
(SUT)
ECOOP, Oslo, June 2004
10
© U2TP Consortium
Agenda
z Motivation
z Basics
of Testing
¾ Overview
z Example
of the UML Testing Profile
Test Specifications
z The
UML Profile and the MOF Model
z The
Mappings
z Concluding
ECOOP, Oslo, June 2004
Remarks
11
© U2TP Consortium
The Testing Profile Roots
• Arbiter
• Validation actions
• Data pools
UML
Testing Profile
Graphical Format
of TTCN-3
UML 2.0
MSC-2000
ECOOP, Oslo, June 2004
UML 1.x
12
SDL-2000
Software Testing
like JUnit, TET, etc.
Protocol Testing
like TTCN-3
• Test control
• Wildcards
• Defaults
• Test components
© U2TP Consortium
Agenda
z Motivation
z Basics
of Testing
z Overview
¾ Example
of the UML Testing Profile
Test Specifications
z The
UML Profile and the MOF Model
z The
Mappings
z Concluding
ECOOP, Oslo, June 2004
Remarks
13
© U2TP Consortium
The Example
OTC Market Makers
Clearing
Company
SWIFTBureau
SWIFTNet
SWIFTBureau
US Bank
Network
EU Bank
Network
US Bank
SSSB Client
ECOOP, Oslo, June 2004
EU Bank
SSSB Client
14
© U2TP Consortium
UML Model of the System Level Archicture
ATM
«import»
HWControl
«import»
Bank
«import»
Money
System
Integration Test
System Test
Unit Test
«import»
«import»
SWIFTNetwork
ECOOP, Oslo, June 2004
15
© U2TP Consortium
Unit-Level Test
ATM
«import»
HWControl
«import»
Bank
«import»
Money
«import»
«import»
SWIFTNetwork
ECOOP, Oslo, June 2004
16
© U2TP Consortium
Unit-Level Test
Money
IMoney
+ fAmount : Integer
+ fCurrency : String
+ equals(m:Object): Boolean
Money
MoneyBag
+ fAmount : Integer
+ fCurrency : String
fAmount : Integer
fCurrency : String
+ Money(a: Amount, c:String)
+ add(m: Money): IMoney
+ contains(m: IMoney): Boolean
+ add(m: IMoney): IMoney
Classes to be
tested
ECOOP, Oslo, June 2004
17
© U2TP Consortium
A Unit-Level Test Suite
„Test“ package
Unit-level test suite
MoneyUnitTest
«TestContext»
«
»
MoneyTest
<<import>>
Money
Bank
<<TestCase>>
addSameMoney():Verdict
addSameMoney()
addDifferentMoney():Verdict
addDifferentMoney()
Unit-level test cases
ECOOP, Oslo, June 2004
18
© U2TP Consortium
A Test Case
Test suite object
performing the test case
sd addSameMoney():Verdict
Class instances
of the SUT
self
Money(20, ”USD”)
«sut»
money1:Money
Money(50, ”USD”)
«sut»
money2:Money
add(money2)
get: money2
add (-) : new Money (70, ”USD”)
return pass
ECOOP, Oslo, June 2004
Return of
test verdict
19
© U2TP Consortium
System-Level Test
ATM
«import»
HWControl
«import»
Bank
«import»
Money
«import»
«import»
SWIFTNetwork
ECOOP, Oslo, June 2004
20
© U2TP Consortium
System Level Test
ATMTest
System-level test
suite with public and
private test cases
«testContext»
ATMSuite
«testComponent»
HWEmulator
-verdict : Verdict
-amount : IMoney
-targetBank : SwiftId
-targetAccount : String
-sourceAccount : String
-pinOk : Boolean
-enteredPIN :
hwCom
String
-message : String
-t1 : Timer
Test component
IHardware
IATM
«import»
«testCase» +validWiring() : Verdict
«testCase» +invalidPIN() : Verdict
«testCase» -authorizeCard() : Verdict
ATM
IBank
«testComponent»
BankEmulator
ECOOP, Oslo, June 2004
-accounts
*
«interface»
IAccount
Miscellaneous
classes
21
© U2TP Consortium
Test Configuration
Coding rule part
SUT part
«testContext»
class ATMSuite
coding
”Encrypted”
«sut»
atm : BankATM
be : BankEmulator
bankCom
atmPort
hwe : HWEmulator
Test component
parts
ECOOP, Oslo, June 2004
current : CardData
Utility part
22
© U2TP Consortium
Test Control (execution of test suite)
Referring test
case
behaviors
sd ATMSuite
ref
verdict = invalidPIN
[verdict == pass]
ref
ECOOP, Oslo, June 2004
[verdict == fail]
verdict = validWiring
23
© U2TP Consortium
Data partition
A Test Case
sd invalidPIN
{readOnly} Integer invalidPIN; { current.isPinCorrect(invalidPIN) == false }
«sut»
atm
hwe
Setting a
timer
current
Stimulus
storeCardData(current)
Stopping
a timer
t1(2.0)
display(”Enter PIN”)
Observation
t1
isPinCorrect(invalidPIN)
Duration
constraint
{0 .. 3}
isPinCorrect(invalidPIN)
isPinCorrect : false
isPinCorrect : false
display(”Invalid PIN”)
Setting
arbitrated
verdict
ECOOP, Oslo, June 2004
display(”Enter PIN again”)
Function
return value
«validationAction»
pass
24
© U2TP Consortium
A Test Case with Default (extracts)
sd validWiring
«sut»
atm
hwe
atmPort
bankCom
Default
application
amount = acceptMoney
wireMoney
default
DisplayDefault
display(”Transaction Accepted”)
selectOperation : true
«validationAction»
pass
ECOOP, Oslo, June 2004
25
© U2TP Consortium
Defaults
Defining an eventspecific default
Applying a componentspecific default
«testComponent
»
HWEmulator
sd DisplayDefault
self
alt
-pinOk : Boolean
-enteredPIN : String
hwCom
-message : String
-t1 : Timer
default
HWEmulator::hweDefault
IHardware
IATM
display(*)
hweDefault
«validationAction»
inconc
Wildcards
?
«validationAction»
fail
ECOOP, Oslo, June 2004
ejectCard /
setverdict(fail)
t1 / setverdict(fail);
*
Defining a componentspecific default
26
display(msg) /
if (msg == ”Connection lost”) then
setverdict(inconc);
else
setverdict(fail);
© U2TP Consortium
System-Integration-Level Test
ATM
«import»
HWControl
«import»
Bank
«import»
Money
«import»
«import»
SWIFTNetwork
ECOOP, Oslo, June 2004
27
© U2TP Consortium
System-Integration-Level Tests
Load tests
SWIFTTest
«testContext»
SWIFTSuite
-numUsers:Integer = 0
-pc:Float
«testCase» -runUSTrxn(p:USTrxnData):Verdict
«testCase» -runEUTrxn(p:EUTrxnData):Verdict
«testCase» -Wiring():Verdict
«testCase» +loadTest
(maxUsers:Integer,p:Float): Verdict
«interface»
Arbiter
«import»
«testComponent»
TransactionController
-initUSbal: IMoney
-initEUbal: IMoney
SwiftNetwork
IHardware
IATM
IAccount
«import»
Bank
default
TransactionController::tcDefault
«import»
ATM
LoadArbiter
«testComponent»
loadManager
-numPass:Integer
-numOther:Integer
«import»
TestData
completed()
ECOOP, Oslo, June 2004
User-defined
arbiter
28
© U2TP Consortium
Test Configuration
SUT parts
«testContext»
«
»
SWIFTSuite
swiftPort
«sut»
euBank : Bank
bankPort
accountPort
eu
swiftPort
«sut»
network :
SWIFTNetwork us
«sut»
usBank : Bank
bankPort
bankPortc
accountPort
bankCom
bankCom
«sut»
usATM: ATM
«sut»
euATM: ATM
atmPort
atmPort
«testComponent»
tc:
transactionController
Test component
parts
Utility part
dp:
dataPool
ECOOP, Oslo, June 2004
bankPort
«testComponent»
«testComponent
»
lm:
loadManager
29
la:
LoadArbiter
© U2TP Consortium
Test Data
TestData
:EUTrxnData[1]
TrxnData
*
account = ”Fred Bloggs”
balance = 10,000
amount = 3500
cardData = Card1
account : String
balance: Integer
amount: IMoney
cardData: CardData
:EUTrxnData[2]
<<<DataPartition>>
EUTrxnData
<<<DataPartition>>
USTrxnData
<<DataSelector>>
getEUTrxnData():TrxnData
<<DataSelector>>
getUSTrxnData():TrxnData
1
«DataPool»
DataPool
:USTrxnData[2]
*
account = ”Barbara Wall”
balance = 10,000
amount = 20
cardData = Card4
<<DataSelector>>
getDistributionInterval():Integer
ECOOP, Oslo, June 2004
:USTrxnData[1]
account = ”Joe Senior”
balance = 10,000
amount = 3500
cardData = Card3
2
2
account = ”Dr Watson”
balance = 10,000
amount = 20
cardData = Card2
30
© U2TP Consortium
Load Test
sd loadTest(int maxUsers, float p)
«testComponent»
lm
lm.T1(testduration)
loop
ref
Wiring(dp)
lm.T2(dp.getDistributionInterval())
lm.T2
lm.T1
«validationAction» pass
ref
ECOOP, Oslo, June 2004
Arbitration
31
© U2TP Consortium
Load Test Contd.
sd Wiring(DataPool p)
«testComponent»
lm
«sut»
usATM
<<create>>
startTest
alt
«sut»
usBank
«sut»
euBank
«testComponent»
tc
ref
runEUTrxn(p.getEUTrxnData())
ref
runUSTrxn(p.getUSTrxnData())
{0..4s}
«validationAction» pass
ECOOP, Oslo, June 2004
32
© U2TP Consortium
User-Defined Arbiter
sd LoadArbiter
numPassed = 0
numOther = 0
setverdict(result)
[result == pass] / numPassed++
setverdict(result)
[result != pass]/ numOther++
Threshold
calculation
Running
completed()
[numPassed/(numPassed+numOthers ) < pc]
[numPassed/(numPassed+numOthers) >= pc]
«validationAction
»
fail
«validationAction
»
pass
Final test
result
ECOOP, Oslo, June 2004
33
© U2TP Consortium
Agenda
z Motivation
z Basics
of Testing
z Overview
z Example
of the UML Testing Profile
Test Specifications
z The
UML Profile and the MOF Model
¾ The
Mappings
z Concluding
ECOOP, Oslo, June 2004
Remarks
34
© U2TP Consortium
Mappings
z To
enable the direct execution of U2TP specifications
z To
reuse existing test infrastructures
¾ Mappings
to
¾ The JUnit test framework
• An open source test technology for Java
• Black-box tests on unit level
¾ The Testing and Test Control Notation, TTCN-3
• A generic test technology by ETSI/ITU-T
• Black-box/grey-box tests on unit, component, integration
and system level
ECOOP, Oslo, June 2004
35
© U2TP Consortium
Mappings
U2TP Specification
Different Mappings
TTCN-3 Modules
TTCN-3 Modules
TTCN-3 Modules
¾
JUnit Test Suites
JUnit Test Suites
Java
...
Different Targets
The mappings define possible, but not the only mappings
ECOOP, Oslo, June 2004
36
© U2TP Consortium
Agenda
z Motivation
z Basics
of Testing
z Overview
z Example
of the UML Testing Profile
Test Specifications
z The
UML Profile and the MOF Model
z The
Mappings
¾ Concluding
ECOOP, Oslo, June 2004
Remarks
37
© U2TP Consortium
At the End:
a Standardized Testing Profile
z
One test notation for many testing applications
z
Universally understood syntax and operational semantics
z
Off-the-shelf tools
z
Cheaper education and training costs
z
Exchange and reuse of test suites
z
Easier maintenance of test suites
¾
Transparency for the test process, increase of the objectiveness of tests and
comparability of test results
Direct support for test design
Integration in the system development process
¾
¾
ECOOP, Oslo, June 2004
38
© U2TP Consortium
Summary
z UML
Testing Profile provides specification means for
test artifacts of systems from various domains
z Enhances UML with concepts like test configuration,
test components, SUT, verdict and default
z Seamlessly integrates into UML: being based on UML
metamodel, using UML syntax
¾ Adopted
at OMG Technical Meeting, Paris, June 2003
http://www.fokus.fraunhofer.de/u2tp/
ECOOP, Oslo, June 2004
39
© U2TP Consortium
Thank you for your attention !
Any further questions ?
© U2TP Consortium