TTCN3
as Another Brick in
the Wall of Test Automation
Authors:
Gergely Horváth (gergely.horvath@nsn.com)
Róbert Vastag (robert.vastag@nsn.com)
TTCN-3 User Conference 2011
7 – 9 June 2011 - Bled, Slovenia
Some Facts About Nokia Siemens Networks
•Joint Venture of Nokia and Siemens
•Started operations on April 1, 2007
•€12.7 billion net sales in 2010
•120+ years of telecom experience
•65,000+ employees
•45,000 service professionals
(including externals)
•75 of top 100 operators worldwide
•150 countries
•3 billion mobile subscribers and ¼ of
world’s voice households served
Agenda
User Plane Routing Development
(what we do)
Agile and TTCN3
(how we do it)
Test Automation with TTCN3
(how TTCN3 is integrated into it)
TTCN3 Core and Fast Test Case Creation
(how it works exactly)
User Plane Routing Development
Control
System Under Test:
Main focus is on user
plane routing
functionalities
Data
TTCN3 executable:
Connected with the
SUT through H.248
interface, simulating
MGW functionalities
RADIO
ACCESS
•
•
•
USER
PLANE
Each interface of the SUT is captured
Radio access is simulated by the traffic
generator
MGWs are simulated by an executable
built from TTCN3 code that is specific
to each test case
User Plane Routing Development
(what we do)
Agile and TTCN3
(how we do it)
Test Automation with TTCN3
(how TTCN3 is integrated into it)
TTCN3 Core and Fast Test Case Creation
(how it works exactly)
Agile and TTCN3
Continuous
Integration
• Automated unit tests
• Scheduled and automated
SW builds
• Automated test execution
environment for regression
(set of functional tests)
Feature
Requirements
Iterative
Feature
Deployment
Agile
Fast
Feedback
• Fast module test environment
for "instant" checking of the
new code
• Nightly functional regression
test set execution
• New feature increment
implemented in every iteration
• Scrum meetings
• Incremental test case design (TTCN3)
Agile and TTCN3
Test Design
• Developing TTCN3 core
functions to follow changes in
user plane routing
• Creating test cases using TTCN3
Test Planning
Unit Test
Test Execution
Evaluation
• Evaluating TTCN3 Results
• Fine tuning the test case
TTCN3 Core Development
& Integration Test
Feature
Requirements
Implementation
Specification
Development
& Unit Test
• MGW simulation by
TTCN3 executable
• Acquiring results
Verification
Feature
Deployment
Automate Tests &
Add to Regression Set
User Plane Routing Development
(what we do)
Agile and TTCN3
(how we do it)
Test Automation with TTCN3
(how TTCN3 is integrated into it)
TTCN3 Core and Fast Test Case Creation
(how it works exactly)
Test Automation with TTCN3
Test Management System
• Tester
• Test Designer
• Test Architect
• TTCN3 Key User
Manager
Test Automation Engine
SVN Repository
for Test Macros
and TTCN3
Test Control Equipment Pool
TTCN3 Build
Server
Test Laboratory
Analysis
Server Pool
Test Laboratory
Test data
from SVN
Configuration
data and
command
sequence
Test Control Equipment Pool
LAB NETWORK
Collecting
execution
results
MGW
simulation
with TTCN3
SUT POOL
Test Automation with TTCN3
Test Management System
• Tester
• Test Designer
• Test Architect
• TTCN3 Key User
Manager
Test Automation Engine
SVN Repository
for Test Macros
and TTCN3
Test Control Equipment Pool
TTCN3 Build
Server
Test Laboratory
Analysis
Server Pool
Test Automation Engine
Test Set Execution Report
Resource Usage Report
Test Execution Report
TTCN3 Result in XML
• Resource usage report
• Resource reservation
• Test set execution report
• Test execution reports
• Archivation of results
• Interface to test management system
• Authorized access (Tester, Test Designer, Test
Architect…)
Test Automation with TTCN3
Test Management System
•Tester
•Test Designer
•Test Architect
•TTCN3 Key User
SVN Repository
for Test Macros
and TTCN3
Manager
Test Automation Engine
• Traceability Support
• Requirement Fulfillment Check
• Maintenance of Test Documents
• Management Reporting
• Statistics
Test Control Equipment Pool
TTCN3 Build
Server
Test Laboratory
Analysis
Server Pool
Test Automation with TTCN3
Roles & Responsibilities:
Test Management System
Tester  Executes and analyzes test
cases, writes incident reports
•Tester
•Test Designer
•Test Architect
•TTCN3 Key User
SVN Repository
for Test Macros
and TTCN3
Test Designer  Creates test cases
according to the test plan
Manager
Automation
Engine
Test Test
Architect
 Responsible
for
regression set creation and high level
planning
TTCN3 Key User  Responsible for
the maintenance of the TTCN3 Core
(common functions used for test case
creation)
Test Control Equipment Pool
TTCN3 Build
Server
Test Laboratory
Analysis
Server Pool
User Plane Routing Development
(what we do)
Agile and TTCN3
(how we do it)
Test Automation with TTCN3
(how TTCN3 is integrated into it)
TTCN3 Core and Fast Test Case Creation
(how it works exactly)
TTCN3 Core System. Why?
H.248 messages have a difficult
hierachy and convey many parameters
Building a template that can match a
specific message is a difficult task, even
for an experienced tester.
Solution:
• New user plane control functionalities are
implemented in TTCN3 core to make testers’
lives easier.
• Only high level parameters need to be given
• Resource Reservation
• Resource Modification
• Resource Removal
• Service Change
• Audit
A Sample H.248 Message:
TTCN3 Core System. How?
Main
Parameters
In Testers’
Hands
+
Complex
Test Steps
=
Specific
Message
Template
Features of Test
Steps:
• Parameterized functions that can construct
templates for H.248 messages
• Compliant with the H.248 standard
• Each test step implements one H.248 transaction
• It hides the complex internal logic from the tester
Actual
H.248
Message
Compare
Pass
Exact
location of
the failure
is shown!
Fail
…and Benefits
of its Usage:
• Only high level knowledge is necessary for
TTCN3 test case creation
• Makes test case creation fast and efficient
• Test cases are well structured
• Global modifications in the functionalities of the
SUT are easy to follow by updating the TTCN3
core (TTCN3 key users’ task)
TTCN3 Library Structure
TTCN core
Feature 1
FEATURE_1_TEST_CASE_1
FEATURE_1_TEST_CASE_2
.
FEATURE_1_TEST_CASE_N
Feature 2
FEATURE_2_TEST_CASE_1
FEATURE_2_TEST_CASE_2
.
FEATURE_2_TEST_CASE_N
Feature N
• Test cases of the same
feature compile into the
same executable
• The executable has to be
started with the test case
ID as input parameter in
order to execute the
proper MGW simulation
TTCN3 Test Case Structure
module FEATURE_1_TEST_CASE_1 {
import from TTCN_CORE all;
Test step
definitions
testcase tc_FEATURE_1_TEST_CASE_1() runs on CORE_TEST_COMPONENT system TEST_SYSTEM
{
SET_CONNECTIONS();
Module
TEST_COMPONENT.start(TEST_SEQUENCE_1());
Connectivity
}
settings
TEST_SEQUENCE_1() runs on TEST_COMPONENT
{
Init();
TEST_STEP_1(PARAM1,PARAM2);
TEST_STEP_2(PARAM1,PARAM2,PARAM3,PARAM4);
TEST_STEP_1(PARAM5,PARAM6);
TEST_STEP_3(PARAM5,PARAM6,PARAM1,PARAM2);
RESOURCE_REMOVAL;
}
}
Actual Test
Case
Behaviour
Conclusion
• The TTCN3 core system makes it easier to cope with frequent
changes in the software that we are testing
• It is thoroughly integrated with the automated test environment
that makes its usage smooth and locating faults easy
• These characteristics aid it in supporting the agile development
• This way we can ensure efficient testing procedures and provide
high quality software products
Thank you!
Your questions are welcome!