The Use of TTCN-3
for Software Testing
Ina Schieferdecker
FOKUS, Berlin
schieferdecker@fokus.fhg.de
Content
•
•
•
•
Overview on TTCN-3
TTCN-3 applied to software
TTCN-3 and XML
Test Execution
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
Content
 Overview on TTCN-3
• TTCN-3 applied to software
• TTCN-3 and XML
• Test Execution
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
Introduction
• TTCN-3 is the Testing and Test Control Notation
• The new standardised test specification and test
implementation language
• Developed from 1999 – 2002 at the European
Telecommunications Standards Institute (ETSI).
• Developed based on experiences from previous TTCN
editions
• Removal of OSI specific concepts; Improvement of concepts;
Introduction of new concepts.
• Applicable for all kinds of black-box testing for reactive
and distributed systems, e.g.,
• Telecom systems (ISDN, ATM, GSM, UMTS); Internet (IP, IP
based protocols and applications); Software systems (Java, XML);
Middleware platforms and component-based systems (CORBA,
.Net, EJB).
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
Overview on TTCN-3
msc mi_synch1_conc1
ASN.1
Types &
Values
mtc
TTCN-3
Core
Language
ISAP1
MSAP2
Tabular
Format
Other Types
& Values 2
:
testcase myTestcase () runs on MTCType system TSIType
{
mydefault := activate (OtherwiseFail);
verdict.set(pass);
:
connect(PTC_ISAP1:CP_ISAP1,mtc:CP_ISAP1);
:
map(PTC_ISAP1:ISAP1, system:TSI_ISAP1);
:
Other Types
PTC_ISAP1.start(func_PTC_ISAP1());
& Values n
PTC_MSAP2.start(func_PTC_MSAP2());
Synchronization();
all component.done;
log(”Correct Termination”);
Graphical
Format
Presentation
Format n
}
:
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
TTCN-3 – Based Black-Box Testing
TTCN-3 Test Case
Port.send(Stimulus)
Port.receive(Response)
• Assignment
of a
Test Verdict
Port
System Under Test
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
Component-Based Test System
SUT
TTCN-3 Test Case
create
start
TC
create
start
ITU-T Workshop, Geneva, Nov. 2002
MTC
TC
TCs
create
start
I. Schieferdecker: TTCN-3
Main Elements of TTCN-3
• Module covers declarations and control
• Templates (test data description) and matching
mechanisms (pattern matching)
• Test configurations
• Formally defined interfaces to the SUT
• Dynamic creation of test component
• Concurrency to describe distributed test setups
• Test cases
• Small (complete) separate compilable programs
• Share (type and data) information
• Test verdicts
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
Content
• Overview on TTCN-3
 TTCN-3 applied to software
• TTCN-3 and XML
• Test Execution
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
TTCN-3 and Software Testing
• Message-Based Software
• Such as Protocols,
Messaging Services, Web
site and Portals, ...
• Example technology is
XML
• Signature-Based Software
• Such as Client-Server,
Peer-to-Peer, Componentbased Systems, ....
• Example technology is
IDL
 Reuse of XML/IDL/... data within TTCN-3
 Define a mapping from XML/IDL/... to TTCN-3
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
TTCN-3 and Software Testing
ASN.1
Types &
Values
Other
IDL Types
& Values 2
TTCN-3
Core
Language
Tabular
Format
Graphical
Format
XML
Other
Types
C, C++,
& Values n
JAVA
ITU-T Workshop, Geneva, Nov. 2002
Presentation
Format n
I. Schieferdecker: TTCN-3
Content
• Overview on TTCN-3
• TTCN-3 applied to software
 TTCN-3 and XML
• Test Execution
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
XML in Distributed Applications
• Increasing number of distributed applications use
XML for
•
•
•
•
Description of messages in datacom protocols
Exchange format between software components
Data description in Web-applications
etc.
• XML
• is a structured method for putting data into a textual
presentation by marking up data
• can have attributes that describe additional information
• describes both attributes and content
• is intuitive and self describing
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
Mapping XML to TTCN-3
• Idea:
Map element tags and attributes to TTCN-3 fields
• Different grammar definitions and mappings
• Schemas
• Embedded approach
• Flat-Catalog approach
• Named Type approach
• DTDs
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
Example
Example – The Dinosaur Database
Paleontologist
feeding the database
ITU-T Workshop, Geneva, Nov. 2002
XML
Student
requesting data
I. Schieferdecker: TTCN-3
The Request Interface
URL
http://www.testingtech.de/TTCN-3_Example/dinolist.xml
XML
XML File
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
Principal Approach
XML
Generation of
test data structure
Adaptor
acc. to the
mapping rules
Web
Service
ITU-T Workshop, Geneva, Nov. 2002
Generation of
test data
Generation of
test behavior
A
D
A
P
T
O
R
Compilation
to Executable Tests
Test
Test
Test
Component
Test
Component
Component
Component
Test
System
I. Schieferdecker: TTCN-3
Example
Structured Type Defintions
<!ELEMENT dinolist (dinosaur*)>
<!ELEMENT dinosaur (name, len, mass, time,
place)>
<!ELEMENT name (#PCDATA)>
<!ELEMENT len (#PCDATA)>
<!ELEMENT mass (#PCDATA)>
<!ELEMENT time (#PCDATA)>
of dinosaur dinolist;
<!ELEMENT type
placeset
(#PCDATA)>
type record dinosaur {
charstring
name,
charstring
len,
charstring
mass,
charstring
time,
charstring
place
}
ITU-T Workshop, Geneva, Nov. 2002
XML DTD
Set of Type
Definition
Record Type
Definition
Field Definition
I. Schieferdecker: TTCN-3
Example
Test Data Definitions
template dinolist DinoList :=
{?, ?, Brachiosaurus, ?, ?, ?, ?};
template dinosaur Brachiosaurus := {
name
:= "Brachiosaurus",
len
:= ?,
mass
:= ?,
time
:= ?,
place
:= ?
}
ITU-T Workshop, Geneva, Nov. 2002
Any Value
I. Schieferdecker: TTCN-3
Example
Test Port
Port Definition
/** communication port type definition */
type port httpTestPortType message {
out url;
in dinolist;
}
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
Example
Test Components
Component
Definition
/** component type definitions */
type component httpTestComponent {
port httpTestPortType httpPort;
timer
localTimer := 3.0;
}
Local Port
Local Timer
type component httpTestSystemComponent {
port httpTestPortType httpTestSystemPort;
}
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
Example
Test Behavior
httpPort.send(requestURL);
localTimer.start;
alt {
[] httpPort.receive(DinoList) {
localTimer.stop;
setverdict(pass);
}
[] httpPort.receive {
localTimer.stop;
setverdict(fail);
}
[] localTimer.timeout {
setverdict(fail);
}
}
ITU-T Workshop, Geneva, Nov. 2002
Sending a
message
Starting
the timer
Alternative
reactions
The expected
response
An unexpected
response
A timeout
I. Schieferdecker: TTCN-3
Example
Simplification: Altstep
altstep DinoList_Default_1()
runs on httpTestComponent {
[] httpPort.receive {
localTimer.stop;
setverdict(fail);
}
[] localTimer.timeout {
setverdict(fail);
}
}
ITU-T Workshop, Geneva, Nov. 2002
Test Component
Type
Handling of
unexpected
response
Handling of
timeouts
I. Schieferdecker: TTCN-3
Example
Simplification: Use of the Altstep
...
activate(DinoList_Default_1());
httpPort.send(requestURL);
localTimer.start;
httpPort.receive(DinoList);
localTimer.stop;
setverdict(pass);
…
ITU-T Workshop, Geneva, Nov. 2002
Default activation
The expected
response is given
here only,
All other cases are
handled by the
default
I. Schieferdecker: TTCN-3
Example
A Test Case
testcase DinoList_Test_1()
runs on httpTestComponent
system httpTestSystemComponent {
Test Case
Definition
MTC Type
TSI Type
map(mtc:httpPort, system:httpTestSystemPort);
activate(DinoList_Default_1());
httpPort.send(requestURL);
localTimer.start;
httpPort.receive(DinoList);
localTimer.stop;
setverdict(pass);
Mapping the
Ports
}
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
Graphical Format
testcase DinoList_Test_1
runs on httpTestComponent system httpTestSystemComponent
mtc
httpTest
httpPort
Component
map(mtc:httpPort,
system:httpTestSystemPort)
activate
(DinoList_Default_1)
LocalTimer
requestURL
DinoList
pass
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
Content
• Overview on TTCN-3
• TTCN-3 applied to software
• TTCN-3 and XML
 Test Execution
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
Test Execution
Test System
IN
Connected Ports OUT
TC2
TC1
OUT
IN
OUT
Mapped Ports
Abstract Test System Interface
OUT
IN
IN
Real Test System Interface
SUT
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
TTCN-3 Execution
• Generic XML adaptor that adheres to the mapping
Test System User
CD: CoDec
CH:
Component
Handling
TM: Management
Test TE
System
SA: Communication
PA: Timer
System Under Test (SUT)
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
TTCN-3 Runtime Interface
• Adaptation to the SUT
Test System User
TRI
CD: CoDec
CH:
Component
Handling
TM: Management
TE
SA: Communication
PA: Timer
System Under Test (SUT)
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
TTCN-3 Control Interfaces
• Adaptation to the test platform/test device
Test System User
CD: CoDec
TCI
CH:
Component
Handling
TM: Management
TE
SA: Communication
PA: Timer
System Under Test (SUT)
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
Summary of TTCN-3
• New version of the only standardized test notation
• Modernization: Programming-like test specification with
flexible data support and various representation formats
• Wider scope of application
• applicable to many kinds of test applications not just conformance
(development, system, integration, interoperability, scalability …)
• applicable in the datacom domain
• Harmonization
• first choice for test specifiers, implementors and users both for
standardized test suites and
• as a generic solution in industrial software development
• Was successfully shown to be applicable to testing of IDL
and XML interfaces, Java and C++ classes, …
• Tools are available
ITU-T Workshop, Geneva, Nov. 2002
I. Schieferdecker: TTCN-3
Thank You.
Questions?