Test Implementation using
TTCN-3 Libraries
TTCN-3 User Conference, 7th of June 2005
1
Stephan Schulz
Sebastian Müller
Validation & Testing Technologies
Computing Architectures Lab
Nokia Research Center
Helsinki
FSCOM
Sophia Antipolis
France
© NOKIA
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
Presentation Outline
• About the ETSI IPv6 Library
• Motivation
• TTCN-3 Libraries
• Library-based Test Suite Implementation
• An Example TTCN-3 Library: ETSI Common
• Conclusions
2
© NOKIA
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
About IPv6 Library
• Implemented at ETSI by STF 276
•
Conformance and interoperability test specs for IPv6
• Joint e-Europe/ETSI project
•
•
Promote deployment of IPv6 technology within Europe
Establish European player in IPv6 testing
• Developed to support IPv6 Logo program by IPv6
Forum
•
Provide faster access to IPv6 market
• Test implementations are publicly (open+free) available
•
•
3
© NOKIA
Insure high deployment
Make tests transparent
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
Motivation
• Common problem in TTCN coding is the “copy &
paste” mentality of test engineers
Lots
of duplicated code + large code size = hard to maintain
• Another problem: “The reinvention of the wheel“
• We desire a test suite implementation which allows as
much reuse of TTCN-3 code within as well as
between different test suites as possible
Much
of the code developed for the IPv6 Logo certification
should be reusable also in a IPv6 test suite for, e.g., 3GPP!
• A test suite should also be useful for additional test
case development by other, external parties
4
© NOKIA
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
The Idea
• Create a test suite using TTCN-3 libraries which can
then be reused to compose also other related test
suites
So
just use TTCN-3 like a regular programming language!
• Ability to reuse depends on making important TTCN-3
code structuring decisions early on in test suite
development
Strong
5
© NOKIA
need for a sound methodology!
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
Benefits of TTCN-3 libraries
• Less test suite specific TTCN-3 code
• Less room for errors in test implementations
• Use of validated TTCN-3 code segments
• “More readable” test implementations
• Less time needed for test case implementation and
validation
• Avoids the “reinvention of the wheel”
 Summary:
6
© NOKIA
Higher quality test suite implementations!
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
Keys to Reusable TTCN-3 Code
• Follow the TTCN-3 Framework
•
•
•
•
Methodology for Test Suite Implementation
Modularization
Rules on how to implement TTCN-3 behavior
Naming conventions
• Specify well defined interfaces
•
Use component type compatibility
• Document TTCN-3 code
•
•
7
© NOKIA
Standard practice in any programming language
Free documentation tools are available
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
Libraries in TTCN-3
• A TTCN-3 library consists of modules separating
•
•
•
behavior(s)
types and constants
templates
• Example: Libraries in the IPv6 test suite
Common IPv6 independent code (e.g, Synchronization)
Common IPv6 code (e.g., Echo Procedure)
Common IPv6 test suite code (e.g., Ipv6 Host Preamble )
But then (not reusable):
• Specific IPv6 test suite code (e.g., IPv6 Core Test Cases)
•
•
•
8
© NOKIA
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
Basic Structuring of IPv6 Test Cases
• Decompose test case behavior based on well
established methodology
•
Configuration (CF) function
•
•
Preamble (PR) function
•
•
get IUT into testing state
Test body (TP) function
•
•
for test configuration set up
perform test as specified by test purpose
Postamble (PO) function
•
return IUT to its initial state
All
of the above implemented by invoking test suite, IPv6, or
common library functions!
• Implement test case or test case (TC) functions by
simply invoking PR, TP, PO functions
9
© NOKIA
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
create/start
preamble sync
(PO) function
(MTC)
(PR) function
(TP) function
(PO) function
(PTC1)
No Test case (TC) functions
needed
10
© NOKIA
(PR) function
(TP) function
(PO) function
(PTCn)
•Specify test case (TC) functions for each
test component
•Use MTC in concurrent test cases only to
spawn and synchronize PTCs
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
(TC) function
(TP) function
(MTC)
(TC) function
(PR) function
testcase
TP sync
testcase
TC functions in Single and Multiple Interfaces
Rules for reusable TTCN-3 Behavior
• Function should not contain any stop statement since
it prevents the execution of postambles!
• Test suite independent functions should avoid to use
the setverdict operation
• Instead use the return value of a function!
• Verdict setting harms reuse, e.g., testing of valid
versus invalid behavior
• A module which defines reusable behavior should
contain a test component type definition tailored its
needs
11
© NOKIA
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
About TTCN-3 Code Documentation
• Without proper documentation reuse is questionable
•
Documentation of library code is well established standard
practice in other programming languages (e.g., man pages)
• It should be possible to use a TTCN-3 library or
modules without ever reading source files
• Any TTCN-3 code should be documented regardless
of its reusability!
•
Guidelines part of framework document
• A free documentation tool t3doc is available:
•
•
•
12
© NOKIA
Follows ideas of similar tools for C and Java, e.g., doxygen
Generates HTML pages from TTCN-3 code comments
WWW tool download: www.farodyne.com\t3doc
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
A Library Example: ETSI Common
• Three modules with common type and value
definitions
•
Common basic, data string and text string constant and type
definitions, e.g., c_CRLF, UInt, etc.
• Module for verdict control handling
•
Definition of function return code and verdict setting
functions
• Module for timing related functions, e.g., f_sleep
• Modules which implement a generic test component
synchronization as well as sync examples
•
13
© NOKIA
Our classic “reinvention of the wheel” 
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
The Synchronization Library Module
• Implements an easy-to-use synchronization handling
for both single as well as multi component test cases
• Key concepts
•
•
Named synchronization points
TTCN-3 default used for handling shutdown
• Uses extensively TTCN-3 log statement
•
Results in more readable execution traces!
• Current main user API:
f_selfSync(syncPointName, retCode)
f_serverSyncClients(#, syncPointName)
f_serverSyncClientsTimed(#, syncPointName, time)
f_serverWaitForAllClientsToStop()
f_clientSync(syncPointName, retCode)
[a_dummyShutDown()] // shutdown example
14
© NOKIA
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
Example of Nice Multi-Component Sync
Ready.
Ready!
f_serverSyncClients(2,“prDone”)
f_serverSyncClients(2,“tpDone”)
Ready. f_serverWaitForAllClientsToStop()
(PTC1)
f_PR_tc1_ptc2()
f_clientSync(“tpDone”,
e_success);
f_PO_tc1_ptc1()
15
© NOKIA
f_tc1_ptc1()
f_TP_tc1_ptc1()
Ready!
Ready. Ready.
(PTC2)
f_PR_tc1_ptc1()
f_clientSync(“prDone”,
e_success);
Ready!
f_clientSync(“prDone”,
e_success);
f_TP_tc1_ptc2()
f_clientSync(“tpDone”,
e_success);
f_PO_tc1_ptc2()
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
f_tc1_ptc2()
Ready!
create/start PTC1 and PTC2
tc1()
(MTC)
Example of Sync Problem Handling
f_serverSyncClients(2,“prDone”)
f_wait4AllClientsToShutDown()
STOP!
STOP!
f_clientSync(“prDone”,
e_error);
a_shutDown_if1()
(calls stop!)
16
© NOKIA
(PTC2)
f_tc1_ptc1()
(PTC1)
f_PR_tc1_ptc1()
Ready!
f_PR_tc1_ptc2()
f_clientSync(“prDone”,
e_success);
a_shutDown_if2()
(calls stop!)
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
f_tc1_ptc2()
STOP!
create/start PTC1 and PTC2
tc1()
(MTC)
Example of Self Synchronization
(MTC)
f_PR_tc2()
f_TP_tc2()
STOP!
f_selfSync(“tpDone”,
e_error);
tc2()
f_selfSync(“prDone”,
e_success);
a_shutDown_if1()
(calls stop!)
• Note that shutdown defaults are neither test case nor
component but only interface specific (e.g, IPv6, SIP, etc.)
• Therefore the same default can be used to shutdown any test
component (which uses that interface) in any test case.
17
© NOKIA
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
Conclusions
• This really works in practice
•
•
The ETSI common TTCN-3 library has already been
successfully deployed within STF 276 as well as a number
of different protocol test suites within Nokia
First versions of the generic IPv6 and IPv6 Core libraries
have been successfully used in the execution of the ETSI
IPv6 test suite prototype against a Linux Ipv6 SUT
• At the end the real key to implementing truly reusable
TTCN-3 code are
•
•
•
18
© NOKIA
TTCN-3 framework
Well-defined interfaces
TTCN-3 code documentation
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
References
• http://www.ipt.etsi.org
• More info about this STF and documents
• Test suite and executable download
• ETSI DTS/MTS-IPT-0001, “Methods for Testing and
Specification (MTS); Internet Protocol Testing (IPT); IPv6
Testing: Methodology and Framework”, Sophia-Antipolis, 2005.
• S. Moseley et al., “IPv6 Test Specifications from ETSI”, Global
Ipv6 Summit, Barcelona, (to appear), June, 2005.
19
© NOKIA
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
About Component Type Compatibility
• Key to create self contained libraries is the runs on
clause with TTCN-3 functions
• An IPv6 preamble may invoke, e.g., a synchronization
function, if its component type definition is compatible
type component Ipv6Comp {
IpPort
pt_ip;
SyncPort pt_sync;
timer
t_sync;
}
type component SyncComp {
SyncPort pt_sync;
timer
t_sync;
}
function f_PR_EchoProcUp(…)
runs on Ipv6Comp {
var FncRetCode v_ret;
v_ret := f_echoProc();
f_clientSync(“prDone”, v_ret);
}
20
© NOKIA
Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller
function f_clientSync(…)
runs on SyncComp {
// some code
}