Security Level:
Experiences of migration
from TTCN-2 to TTCN-3
Prasanna Venkatesh
Balakrishna S Pol
Murulidhara R
Huawei Technologies India Private Ltd
HUAWEI TECHNOLOGIES CO., LTD.
www.huawei.com
First a glimpse of history
• TTCN-2 (formally TTCN-2++) is defined as per ETSI TR
101 666. This standard which was a significant revision
on the original TTCN was released in 1999.
• However by this time the need for a more
“programming language” oriented TTCN had been
considered and analyzed. By 2000 ETSI completed the
recommendations for TTCN-3. These were adopted as
the ES 201 873 series.
But by far TTCN-3 is the most successful of the TTCN
standards and has dominated protocol testing for
more than a decade.
HUAWEI TECHNOLOGIES CO., LTD.
Is TTCN-2 relevant today?
Well, may not be for many of you!
• But there are a large set of legacy test cases which still run
on TTCN-2.
• In our team in Huawei, we roughly have 20% of our test
cases in TTCN-2 and this count seems to be growing!
TTCN-2 Test case count (Protocol-wise)
HUAWEI TECHNOLOGIES CO., LTD.
TTCN-2 Test case increase % in last 5 years
This topic has been discussed in T3UC
in the past!
Couple of past T3UC papers
1. Comparing TTCN-3 and TTCN-2, Claude Desroches, Strategic
Test Solutions (T3UC 2004)
2. Refactoring and Converting a TTCN-2 Test Suite, Thomas
Deiß, Nokia (T3UC 2005)
But the issue of TTCN-2 to TTCN-3 conversion is still a relevant
issue for us. So we thought that we can discuss this issue once
again and with the Indian T3UC community.
Is this a relevant issue for you?
HUAWEI TECHNOLOGIES CO., LTD.
Why at all migrate to TTCN-3?
• TTCN-3 is superior to TTCN-2
• TTCN-3 standards are continuously enhanced
and have better support in the industry
• TTCN-3 professionals are more easier to get
than TTCN-2 professionals
• TTCN-3 boasts a greater number of tools,
better quality tools and wider functionality
tools.
HUAWEI TECHNOLOGIES CO., LTD.
Some more practical reasons
• Debugging a TTCN-2 script is a nuisance, there are tools which can
debug TTCN3 script just like C code debugging.
• TTCN-3 offers a rich set of predefined function will make scripting
easy.
• Editing in tabular format is a slow process compared to one done
using TTCN3 text
• Allows dynamic test configurations, there by adopting for a variety
of testing
• Extensible. anything can be represented using TTCN3, hence can be
tested.
• Very less help in terms of online groups, problem experience etc in
resolving complex issues
• Easy to train new members to work with TTCN3 as it is more like
programming language.
HUAWEI TECHNOLOGIES CO., LTD.
Approaches for migration
1
2
3
Retain the existing
test cases in TTCN2. Write only the
new ones in TTCN-3
Rewrite or Refactor
all the test cases to
TTCN-3. Generally
the framework is
redesigned and use
the advantages of
TTCN-3
Perform a language
level conversion to
TTCN-3 from TTCN2. Use tools or
automated
approaches to
perform this
conversion
HUAWEI TECHNOLOGIES CO., LTD.
Only new test cases on TTCN-3
• If a few new test cases have to be written, only these test
cases can be written in TTCN-3 while retaining the old ones
in TTCN-2. The ASN.1 definitions can be taken from TTCN2, but have the TTCN-3 test cases as a separate suite.
Advantages of the approach
Issues with the approach
Old test cases continue to run
There could be duplication of test
functionality
Less effort consumed
Most likely two tools would be
required to execute the test suites.
Summary: For situations where there is a totally independent new functionality to
be tested, this approach is good. Additionally if the new test cases are roughly 2030% of the existing test cases this approach is good.
HUAWEI TECHNOLOGIES CO., LTD.
Technical Advantages of TTCN-3 in brief
• TTCN2 is adequate for conformance test but lacked provisions for new and
more different test needs like interoperability testing, system testing,
integration testing
– Also there was limited scope for user in terms of encoding, test control etc.
Hence TTCN3 contains,
– Syntax similar to programming language with the Tabular Notation and MSC as
presentation languages.
– Dynamic system configuration
– Control of Test execution mechanisms
– Improved encoding formats.
– operations for synchronous and asynchronous communications
– combined use of TTCN-3 and ASN.1 (and potential use with other languages
such as IDL)
– User-defined attributes
• TTCN-3 also offers enhanced modularity and hence test case reusability.
HUAWEI TECHNOLOGIES CO., LTD.
Refactor/Rewrite to TTCN-3
• If effort could be afforded the best way to go about is the
refactor/rewrite all TCs from TTCN-2 to TTCN-3. In this
approach the advantages of TTCN-3 is used. ASN1.
definitions, part of the core TTCN-2 logic can be reused.
Advantages of the approach
Issues with the approach
Complete use of TTCN-3 capability
Effort intensive
Improved readability/maintainability
of the test suite.
It is possible that the core test case
logic gets disturbed during the
rewrite process.
Summary: If there is a small set of test cases and the test cases in TTCN-2 are well
understood then this approach is the best approach. Alternatively if a new set of
test cases have to written which larger than the existing set, then this approach
should be considered.
HUAWEI TECHNOLOGIES CO., LTD.
Language level conversion to TTCN-3
• If the suite complexity is lower, then a tool + some manual
correction can migrate it to TTCN-3. This approach uses
standard conversion model defined by the ETSI TTCN2:TTCN-3 mapping. Few tools are available to do this job.
Advantages of the approach
Issues with the approach
Less efforts required for conversion
compared to refactoring approach
Test suite after conversion could become
un-maintainable
Quite a few commercial tools are
available in the market
Test case logic could get distorted, leading
to lower quality assurance
Knowledge of the language is sufficient,
test case logic need not be understood.
Summary: If there is a huge number of test cases (with lesser complexity) in TTCN2 and conversion need to be finished in less time then this approach is the best
approach.
HUAWEI TECHNOLOGIES CO., LTD.
Aren’t TTCN-2 & TTCN-3 supposed to
compatible?
They are to a great degree compatible!
• But that does not mean, you can pick a TTCN-2 test case and run it
on a TTCN-3 tool.
• ETSI TR 101 874 specifies the mapping between TTCN-2 and TTCN-3
• ETSI specifies how a each element in The TTCN2 tabular format is
converted in to TTCN3 pro forma.
– But the conversion specification for the TTCN2 tabular format to
TTCN3 core language is not clear.
– It is essential for user to know the TTCN2 conversion in terms of TTCN3
core language. Most modern users are well aware of core language
than tabular format. The ease for learning TTCN3 core is far better
compared to TTCN3 TFT concepts.
HUAWEI TECHNOLOGIES CO., LTD.
Schematic approach towards conversion
• The flow depicts the
simplified conversion
approach by converting
each TTCN-2 construct into
respective TTCN-3 core
language constructs
– This flow uses an element to
element mapping of TTCN-2&
TTCN-3 to perform the
conversion.
– Hence this approach will not
lead to an optimized TTCN-3
test suite.
HUAWEI TECHNOLOGIES CO., LTD.
Some Industry tools available for migration
1.
Elvior – TestCast
http://www.elvior.com/t2-to-t3
2.
Testing Technologies
TTTwo2Three http://www.testingtech.com/products
/tttwo2three.php
3.
IBM Rational System Tester
(t2tot3 converter) - http://www01.ibm.com/software/awdtools/tteste
r/
4.
OpenTTCN3 http://www.openttcn.com/products/t
ester2012
HUAWEI TECHNOLOGIES CO., LTD.
Challenges involved
The complexity of conversion depends on how TTCN-2 test suite is designed
and organized.
Following are few challenges we faced
•
TTCN-2 test suite uses many global variables (like TSP & TSO). But in TTCN3 there
are no global variable concept which leads to difficulty in conversion
•
When a tool is used for conversion, there were a huge number of compilation
errors which took a lot of effort to solve them. The number of errors depends on
the complexity of the TTCN2 suite.
•
All the TTCN-2 test cases, steps, ASN definitions and PDUs are maintained in single
.mp file. It is very tedious to modularize these into TTCN3 .3mp files
•
When a tool is used for conversion, the ASN structures which again linked to other
ASN definition are very difficult to understand and maintain
HUAWEI TECHNOLOGIES CO., LTD.
Examples of conversion
TTCN2
HUAWEI TECHNOLOGIES CO., LTD.
TTCN3
Key points and learnings
• If you have a growing TTCN-2 suite, it is better to think
about a migration strategy to TTCN-3.
– Else it leads to more cost and trouble in the future
• The migration strategy is based on the actual
conditions of the project.
– There are three kinds of migrations
1. New test cases in TTCN-3
2. Refactor/Rewrite existing test cases to TTCN-3
3. Use a tool to do language conversion to TTCN-3
•
Using a tool to convert is also generally not so
straight forward, but it is probably the most optimal
way.
HUAWEI TECHNOLOGIES CO., LTD.