TTCN, Past, Present
and Future
O. Monkewich
Chairman, ITU-T Joint Coordination
Activity on Conformance and
Interoperability Testing (JCA-CIT)
Overview



Past - some pre-history, before TTCN-3
Present - new ITU initiative
Future - how can TTCN-3 and TTCN-3
experts help
In the Beginning

OSI


Implementations from different suppliers to interconnect
October 10-13, 1983, Ottawa, Canada,
ISO/TC97/SC16/WG1


Dave Rayner of NPL, UK, appointed Rapporteur of OSI
Conformance Testing Methodology and Framework
First meeting achievements
 Static Conformance – visual review
 Dynamic Conformance – in an instance of communication
 Will need a test notation


Test specifiers to be allowed their own notation
The standard would provide some examples
Test methods, Verdicts, Multilayer, Single-layer, Embedded
testing

February 13-15, 1984 GMD Darmstadt,
Germany ISO/TC97/SC16/WG1 Conf. Group

First discuss of requirements for test notation



ASPs, PDUs, Test Methods
Verdicts
Must consider



single-layer testing
embedded single-layer testing
multi-layer testing
Neutral test notation or an
FDT?

November 12-16, 1984, Ft. Lauderdale
Florida, ISO/TC97/SC16/WG1 Conformance

Anthony Wiles, Uppsala University, Sweden



Uppsala University developing tools for testing
Conformance Group developing a neutral test
notation but is also looking at FDTs
Plan to meet with the ISO FDT experts

Prepare testing examples for comparison with other
potential notations
Tables, Trees and FDTs

February 3-13, 1985, Paris, France, ISO/TC97/SC16
 France - a table per test case
 UK proposed behaviour trees – textual, branching with indentations
 Both adopted – birth of TTCN
 Met with FDT groups - LOTOS, Estelle
 Continue with TTCN but keep looking at FDTs

July 7-12, 1985, Manchester, UK, ISO/TC97/SC16 Conformance
Rapporteur Group
 Upper Testers, Test Coordination Procedures
 Requirements on test notation:
 Verdict assignment: Pass, Fail, Inconclusive
 Valid, Invalid and Inopportune PDUs
 timing, addressing
 sending, receiving PDUs
 Compare observed vs. required outcome
The “Bows-and-Arrows” notation

October 18-30, 1985, Paris, France, ISO/TC97/SC6
Conformance test suites




3 X.25 test suites in ad hoc time sequence diagrams
 DTE-DCE physical link – computer-modem
 DTE-DCE link access to network – computer-modem
 DTE-DTE packets over network – computer-computer
Canada argues for using TTCN, labels the US notation “bowsand-arrows” notation
Canada, please translate to TTCN or …
Initiated effort to develop the University of Ottawa/Nortel
“Workbench” tool for TTCN
Stabilize TTCN for X.25, MHS

April 21-29, 1986, ISO/TC97/SC21 jointly with
CCITT Q.47/VII, San Diego (La Jolla), California





Should ASN.1 be added for PDU description
Canada wants TTCN syntax to expressed in BNF
TTCN is informal - sufficient to define it by example
CCITT (now ITU-T) will develop a test suite for MHS, they
prefer TTCN
A version of TTCN was stabilized (San Diego version)
TTCN still informal, but, can
we describe it in BNF

September 8-26, 1986, Egham, UK,
ISO/TC97/SC21




Canada argues need BNF for machine readability
Keep San Diego (La Jolla ) version of TTCN for X.25 and MHS
LOTOS gaining interest
Criteria for test notation identified




Notations not evaluated by February 1987 will not be considered
TTCN is still considered informal: it is for human readability,
abstract test suite specifiers, not for test realizers
PIXIT introduced (tongue-in-cheek acronym) but was adopted
Test suite work not for the conformance group, shifted to protocol
specifier groups
X.25 Test Suites will Allow Other
Test Notations (in an Annex)

October 13-24, 1986, Tokyo, Japan
ISO/JTC1/SC6 Conformance testing


Definition of any new test notation will be added
to the test suite annex
Canada promises X.25 Packet Level test suite in
TTCN to the next meeting
BNF for TTCN arrives

February 3-12, 1987, Nice, France,
ISO/JCT1/SC21/WG1 Conformance
Rapporteur


April 27-May 1, 1987, Washington, DC,
TTCN meeting


Canada tabled BNF description of TTCN, TTCN.MP to come
Construct to permit TTCN trees to execute in parallel introduced
February 28-March 9, 1988, McLean,
Virginia, USA, ISO/JCT1/SC21/WG1

TTCN should be separate part of DP9646 rather than an annex
of DP9646 Part 2
TTCN.MP



May 23, 1988, Teddington, NPL, UK, TTCN Editing meeting
July 14-22, 1988, Stockholm, Sweden, SO/JCT1/SC21/WG1
Conformance editing meeting (hosted by Telia
Research/Teletest)
 Teddington/Stockholm TTCN becomes Part 3 of DP9646
(DP9646-3)
 TTCN made self-consistent, integrated with BNF, machine
readable TTCN.MP
 Could not agree on Operational Semantics
November 28-December 8, 1988, Sydney, Australia ISO/IEC
JTC1/SC21/WG1 Conformance Group
 DP9646 Part 3 did not make it to DIS
 Operational Semantics substantially changed
 Stable “kernel” of TTCN could go to DIS after the next Stockholm
editing meeting, July 18-29, 1989.
TTCN extensions

February 7-17, 1991, Phoenix, Arizona,
ISO/JCT1/SC21/WG1 Conf. Group


TTCN extensions
 Rational for concurrency, TCs, PTCs and MTCs
 Coordination Messages, Coordination Points, Test
Configurations
 Starting and stopping Test Components CREATE and
TERMINATE
May 20-31, 1991, Arles, France, ISO/JCT1/SC21


Editing TTCN Extensions
 DEFAULT ACTIVATE, DEACTIVATE, RETURN,
OTHERWISE
 Handling background activity (cannot be forced or ignored)
TTCN-1 was considered completed
TTCN-1 approved as ISO/IEC
International Standard

Ottawa, February 18-19, 1992,
ISO/IEC/JTC1/SC21/WP1



TTCN-1 (TTCN edition 1) approved as ISO/IEC International
Standards and published on 12 November 1992
Anthony Wiles hands over TTCN Editorship to Os
Monkewich
Further Extensions as well as the ongoing Extensions
Concurrency: Draft Amendments
One and Two to TTCN-1:

RTP North Carolina, 19-27 October, 1992,
ISO/IEC/JTC1/SC21/WP1 TTCN Editing
Meeting

Extensions and Further Extensions






TC, PTC, MTC
Test Suite Operations and control agreed
CREATE(List of PTCs) passed by value
RETURN for default trees only
?DONE added
TERMINATE deleted
The Last TTCN-2 Meeting

BSI, Chiswick, London, UK,
September, 1994,
ISO/IEC/JTC1/SC21/WP1 TTCN Editing
Meeting



Cleanup, over 200 comments, defects reports
on Extensions and Further Extensions
Modular TTCN – Southampton, July 1994
Remaining work done by e-mail
TTCN Edition 2 (TTCN-2)

TTCN-1 vs. TTCN-2

TTCN-1 Non-concurrent:




Maximum of two PCOs
Single service provider
Behaviour is expressed successive branch
indentations only
TTCN-2 Concurrent





Unlimited number of PCOs
Multiple service providers
Behaviour expressed by parallel execution of
behaviour trees
Modules for reuse of ATSs
Uses 54 different tables
Years after the beginning




Started in 1983
TTCN-1 completed and published in 1992
TTCN-2 completed and published in 1998
ISO/IEC JTC1 stops work on OSI



Defect Report processing more difficult
ETSI corrected defects to produce TTCN-2++
ITU-T SG10 and SG7 (now SG17)



Languages, SDL, MSCs, ASN.1 and TTCN
Adopted TTCN-2++ in 2002 as Rec. X.292
2001- 2011, SG17 approved and continues to updated 11
TTCN-3 Recommendations based on ETSI standards
Since 2002, ITU-T has approved
11 TTCN-3 Recommendations











Z.161: TTCN-3 core language
Z.162:TTCN-3 tabular presentation format (TFT)
Z.163: TTCN-3 graphical presentation format (GFT)
Z.164: TTCN-3 operational semantics
Z.165: TTCN-3 runtime interface (TRI)
Z.166: TTCN-3 control interface (TCI)
Z.167: TTCN-3 mapping from ASN.1
Z.168: TTCN-3 mapping from CORBA DL
Z.169: TTCN-3 mapping from XML data definition
Z.170: TTCN-3 documentation comment specification
Z.171: TTCN-3 and the use of C/C++
New Horizons

ITU-T WTSA-8 Resolution 76 approved


ITU Mark Program


TSB to carry out specific tasks with the view to
introducing the use of ITU Mark Program
permit manufacturers and service providers to add a
visible ITU mark to their product if it conforms to ITUT Recommendations
Assist the Developing Countries


Capacity building
Establishing of regional test laboratories
Interoperability – the ultimate
objective

Interoperability does not follow from
conformance

Today’s ITU-T Recommendations not good
enough for global interoperability





No explicit conformance or interoperability
requirements
Interoperability is achieved by means of on-the-fly
adjustments
Adjustments not made publicly available
Interoperability exclusive
Developing Countries want both

Conformance and Interoperability
Role of TTCN-3

Conformance testing is required to



Qualify for the ITU Mark
Be entered into the ITU Database
Testing quality




Must support the criteria for formal testing
Test laboratory accreditation
Criteria for Global harmonization and recognition
Product certification


Voluntary – wide acceptance in the marketplace
Regulatory – meet requirements of national law
Can anything go wrong?

ITU Recommendations for the modern times






Packaging of conformance requirements
 specifications from a standard implemented in different boxes
 no declaration of which requirements are in what box
 boxes may communicate over proprietary interfaces
 no standardized interfaces or access points for testing
Boxes and network components from multiple vendors
Streamed video utilizing a variety of "hidden" codecs
Operators and service providers downloading their software
onto user devices
Plug-ins downloaded for use with the implementation
Hidden proprietary actions as providers make changes to the
user system
CIT Section for ITU
Recommendations

Proposal to include requirements for




Reporting any factors needed for achieving
conformance and interoperability
Defining the essential elements for end-to-end
communication
Information for network element management for
multi-vendor mix of network component
JCA-CIT draft CIT Section and Guide


Use Guide to complete the CIT Section
Include Section in each Recommendation
Joint Coordination Activity-Conformance
and Interoperability Testing (JCA-CIT)

Membership




Meet 3 to 4 times per year


Combination of in body, via Webinar or GoToMeeting
Focus



TSB staff and Study Group chairmen and vice-chairmen
Representatives appointed by the Study Groups
Experts form A.4/A.5 SDOs such as ETSI
Recommendation ingredients for testing
Assistance in the implementation Resolution 76
 Roadmap for implementation
 ITU Mark, ITU Database, capacity building, test laboratories
URL: http://www.itu.int/en/ITU-T/jca/cit/Pages/default.aspx
New opportunities for experts
Training in Developing Countries







Conformance, compliance and interoperability
Production of Requests For Proposal (RFPs)
procurement of products
Establishing new test laboratories
Test laboratory accreditation
Product certification
On-site, hands-on training
New test laboratories
Establishing regional test laboratories

Setting up a test laboratory from start

Defining the requirements for the lab
Mission statement
Business plan










laboratory operation
cost recovery
Funding
land purchase
Specification of the equipment needed for its
operation
Scope of what will be tested
Plans for accreditation
Contribution to standards
Conformance and interoperability




Improving Recommendations for conformance
and interoperability
Development and standardization of test suites
The testing process
Recognition criteria



Assertion of compliance
Credentials for admission to be listed by ITU
Conclusions




15 years to develop early concepts used in TTCN-3
10 years of TTCN-3
New ITU and Regional initiatives creating new
opportunities for TTCN-3 and its experts
Testing experts are needed to







Develop and improve standards to allow testing
Help Developing countries gain technical capabilities
Help build new test laboratories in regions
Wide recognition of test results
Accreditation and Certification
ITU Database population
ITU-Mark management