UML Testing Profile
Graphical Testing With UML
The UML Testing Profile
Eric Samuelsson
eric.samuelsson@telelogic.com
© Telelogic AB
UML Testing Profile
Agenda
• Background
• Key concepts
• Advanced concepts
• Summary
© Telelogic AB
UML Testing Profile
Background
• Test technologies are not on par with modern design
technologies such as model-driven architectures (MDA)
• There’s a need for a widespread graphical test language
• OMG realized this and issued and RFP for a graphical testing
profile to UML
– Using the UML ensures widespread use across a wide range of
application areas
– Numerous tool vendors support UML
– UML is a proven de facto standard
– But UML is focused on definition of system structure and behavior
and has no built-in testing constructs...
© Telelogic AB
UML Testing Profile
UML Testing Profile RFP
• Issued by OMG in July 2001
• LOI’s submitted by
– Telelogic, Rational, SofTEAM, FOKUS,
Ericsson, Motorola, IBM
• One joint submission
• Initial submission made in April 2002
• Revised initial submission made in June 2002
– Terminology changes
– Elaboration of Test Data
• Final submission in January 2003
© Telelogic AB
UML Testing Profile
Requirements and restrictions
• Define structural and behavioral test concepts
• Conformance testing only
– Black-box functional testing
• Based upon the UML meta-model
– UML 2.0 as proposed by the U2 Partners
– Inter-operation with existing testing technologies
– TTCN 3
– Junit
– ...
© Telelogic AB
UML Testing Profile
Current proposal
• A graphical testing language based the current UML 2
metamodel
• Reusing as many UML concepts as possible
– Recognition important for UML users
– No need to re-invent the wheel
• Designed with TTCN 3 in mind
– Semantics basically follows TTCN 3 semantics
– But caters for other technologies like JUnit as well
© Telelogic AB
UML Testing Profile
Current proposal
•
Defined as a UML profile
– Will be recognized by UML users
•
Basically a number of stereotypes
– Easy to implement for tool vendors
•
Models will be interchangeable
– Any tool implementing XMI will be able to exchange models
UML
«import»
«profile»
Testing Profile
«metaclass»
BehavioralFeature
«stereotype»
TestCase
© Telelogic AB
UML Testing Profile
Key Concepts
• Test Architecture
– Test Context, Test Configuration
– Test Component, SUT
– Arbiter
• Test Behavior
– Test Case
– Verdict, Validation Action
– Defaults
• Test Data
– Logical partitions, Pattern matching
• Test Deployment
© Telelogic AB
UML Testing Profile
Test Objective
• An informal description of what to test in one test case
• Commonly expressed using natural language
”Verify that if a user inserts and authorizes a
valid card correctly, he is able to withdraw
money if he has sufficient funds.”
• Can also be expressed using other diagrams, such as
use case diagrams and sequence diagrams
Withdrawal
Client
Client
<<include>>
Tester
InvalidPin
© Telelogic AB
UML Testing Profile
Test Case
• A specification of one test, including
– Test objective
– Test behavior
• Defined as a BehavioralFeature in UML
• It’s method is the behavior of the test case
«testCase» +validWithdrawal() : Verdict
«testContext»
ATMContext
«testCase»
+validWithdrawal() : Verdict
+invalidPIN() : Verdict
-authorizeCard() : Verdict
© Telelogic AB
UML Testing Profile
Test Behavior
• Describes the expected behavior of a test case
• Defined using any type of UML behavior
– Interactions, statemachines and activities
• Facilitates specification of arbitrarily complex behavior
sd ATMContext
«sut»
atm
hwe
ref
invalidPIN()
Idle
storeCardData(currentCard)
display(”Enter PIN”)
[verdict == pass]
[verdict == fail]
WaitCode
isPinCorrect(enteredPIN)
ref
validWithdrawal()
true
© Telelogic AB
UML Testing Profile
Test Architecture
• A complete specification of all types used in the tests
– An ordinary UML package
– Contains Test Contexts, Test Components, Ports, Interfaces, etc.
«testArchitecture»
ATMTest
© Telelogic AB
UML Testing Profile
Test Architecture Example
«testArchitecture»
ATMTest
IAccount
«testContext»
ATMContext
«testComponent»
HWEmulator
bePort
«testCase»
+validWithdrawal() : Verdict
+invalidPIN() : Verdict
-authorizeCard() : Verdict
-pinOk : Boolean
-enteredPIN : String
-message : String hwCom
«testComponent»
BankEmulator
*
IScreen, ICardMachine,
IMoneyBox
IATM
accounts
Account
balance : Integer
number : String
credit(a : Integer)
debit(a : Integer)
ATM::CardData
*
cards
pinCode : Integer
number : String
isPinCorrect(c: Integer) : Boolean
© Telelogic AB
UML Testing Profile
Test Component
• Implements (parts of ) the behavior of one or more test cases
• Each test case is normally executed by several components
• Ordinary classes or components
«testComponent»
HWEmulator
-pinOk : Boolean
-enteredPIN : String
-message : String hwCom
0..1
IScreen, ICardMachine, IMoneyBox
IATM
-currentCard
ATM::CardData
pinCode : Integer
number : String
isPinCorrect(c: Integer) : Boolean
© Telelogic AB
UML Testing Profile
Test Context
• A classifier that contains
– a number of test cases and their behavior and objectives
– default behavior used by the test cases
– configuration of test components
– test control behavior
• Can be a class, a component or a collaboration
«testContext»
ATMContext
«testCase»
+validWithdrawal() : Verdict
+invalidPIN() : Verdict
-authorizeCard() : Verdict
© Telelogic AB
UML Testing Profile
Test Configuration
• A specification of the structural aspects of test cases
• Defines the parts participating in the test case and their
interconnections
– Test Components, SUT, Ports, Interfaces and Connectors
• Defined as the internal structure of the Test Context
«sut»
atm : BankATM
be : BankEmulator
netCom
bePort
atmPort
hwCom
hwe : HWEmulator
accnts : Account [0..*]
© Telelogic AB
UML Testing Profile
Test Control
• Test execution can be controlled is two ways
– By defining the classifier behavior of a test context
– Or by invoking the test cases from outside of the context
© Telelogic AB
UML Testing Profile
Test Control by ClassifierBehavior
sd ATMContext
ref
invalidPIN()
«testContext»
ATMContext
«testCase»
+validWithdrawal() : Verdict
+invalidPIN() : Verdict
-authorizeCard() : Verdict
[verdict == pass]
ref
[verdict == fail]
validWithdrawal()
© Telelogic AB
UML Testing Profile
Test Control by Invocation
ad myOp
«testContext»
MyTestContext
MyClass
-verdict : Verdict
+myOp()
verdict := context.TC1()
-context
*
«testCase»
+TC1() : Verdict
+TC2() : Verdict
-TC3() : Verdict
[verdict == pass]
[else]
verdict := context.TC2()
© Telelogic AB
UML Testing Profile
Test Data
• Parameters, default values, return values are reused from UML
without changes
• Logical partitions
• Pattern matching
• TDB
© Telelogic AB
UML Testing Profile
Test Deployment
• Test Deployment is important, but there is probably no
difference between deploying test systems compared to
ordinary UML systems.
• We will try to reuse the existing UML deployment concepts
without changes
• Under investigation
© Telelogic AB
UML Testing Profile
Advanced Concepts
•
Test Evaluation
– Verdict
– Validation action
– Arbiter
•
•
Default behavior
Structural and behavioral reuse
© Telelogic AB
UML Testing Profile
Test Evaluation
• A number of concepts are used to evaluate test results
– Verdicts
– Validation actions
– Arbiter
© Telelogic AB
UML Testing Profile
Verdicts and Validation Actions
•
•
•
Each test component is
associated with a verdict
«sut»
Client
Validation Actions are used
to set the verdict
A Validation Action sets the
verdict of a test component
and reports the verdict to
the Arbiter
ATM
storeCardData(cardData)
display(”Enter PIN”)
isPinCorrect(pin)
alt
true
false
setverdict(fail)
© Telelogic AB
UML Testing Profile
Arbiter
• A test component that knows how to evaluate the test behavior
• Separates test evaluation from test case specification
– One behavior can yield different results
• Each test context has a default arbiter
– Implicit connections to all test components
– TTCN 3 semantics
• Advanced arbiters can be defined
– Caters for more advanced semantics
«arbiter»
myArbiter
setVerdict(v : Verdict)
© Telelogic AB
UML Testing Profile
Default behavior
•
•
•
Defines behavior for unexpected observations
Can be expressed using any UML behavior
Can be applied to several different levels:
– Test component
sd authorizeCard
– Message reception
– Alternative statement
«sut»
Client
– ...
ATM
storeCardData(cardData)
enterPinDefault
display(”Enter PIN”)
isPinCorrect(pin)
true
© Telelogic AB
UML Testing Profile
sd enterPinDefault
self
alt
getStatus
statusOk
Default expressed
as interaction...
setverdict(pass)
display(”Connection lost”)
setverdict(inconc)
ejectCard()
setverdict(fail)
© Telelogic AB
UML Testing Profile
enterPinDefault
getStatus / ^statusOk, setverdict(pass);
...and as
statemachine
ejectCard /
setverdict(fail)
*
display(”Connection lost”) /
setverdict(inconc)
The representations are semantically equivalent
© Telelogic AB
UML Testing Profile
Meta Default
• Each test component has a meta default to cater for
differences in semantics
• Can be redefined
MetaDefault
*
* / defer
© Telelogic AB
UML Testing Profile
Reuse of Test Structure and Behavior
© Telelogic AB
UML Testing Profile
«testContext»
ATMContext
«sut»
atm : BankATM
be : BankEmulator
netCom
accnts : Account [0..*]
bePort
atmPort
ATM
hwCom
hwe : HWEmulator
ac: AuthorizationContext
Client
© Telelogic AB
UML Testing Profile
sd ValidWithdrawal
«sut»
atm
hwe
atmPort
ref
be
netCom
ac.authorizeCard()
selectOperation(withdrawal)
true
withdraw(amount)
findAccount(cardData)
account
debitAccount(account, amount)
true
deliverMoney(amount)
display(”Take cash”)
true
© Telelogic AB
UML Testing Profile
Summary
• The testing profile enables complete graphical specification of
tests for conformance testing of UML models
– Defined as a UML profile
– Reuses existing UML concepts whenever possible
– Designed for inter-operation with TTCN-3 and other technologies
• Due date
– January 6th, 2003
• More information
– Testing Profile Consortium
– OMG’s Testing Profile page
© Telelogic AB