DOCUMENT #:
GSC15-PLEN-51
FOR:
Presentation
SOURCE:
ITU
AGENDA ITEM:
Plenary 6.10
CONTACT(S):
Reinhard.Scholl@itu.int
Conformance and Interoperability
Testing
Reinhard Scholl
ITU
Global Standards Collaboration (GSC)
GSC-15
Highlight of Current Activities (1/3)
 ITU-T SG 11: Lead study group on test
specifications
 ITU-T SG 15: Has extensive testing ecosystem
 ITU-T SG 16: its IPTV standards (H-series) was
put to test in ITU-T’s first interop event (20-23 July)
• 3 IPTV testing specs approved; others further developed
 ITU-T SG 17: Lead study group on languages and
description techniques
• Also: Q14/17 on Testing methodology, specification language and
Generic test suites for CIT
2
Highlight of Current Activities (2/3)
 ITU-T JCA-CIT: Joint Coordination Activity on
Conformance and Interoperability Testing
• Works with all Study Groups to:
 address the topics relevant to conformance and
interoperability testing
 enhance the quality of ITU-T Recommendations
• Eliminate errors, ambiguities unverified behaviours
• Include sufficient detail for implementation and testing
• Include interfaces needed for interconnection and testing
 develop methodologies and frameworks for testing
3
Highlight of Current Activities (3/3)
 Regional ITU consultations on conformance
assessment and interoperability
• Quito, 8 July 2010
• Nairobi, 30-31 July 2010
• Sydney, 16-17 Sep. 2010
 Pilot conformity database developed
 ITU Interop started
• IPTV (July in Geneva; Sep. in Singapore; Dec. in India)
• Home networking
4
Strategic Direction
 WTSA-08 Res. 76 on conformance & interop
 … supported & complemented by WTDC-10
Res. 47 (ITU-D)
 ITU Council-09 mandated
•
•
•
•
a pilot conformity assessment database
interoperability events
human resources capacity building
assistance in the establishment of test facilities in
developing countries
• a business plan for long-term implementation
5
Challenges
 Regional ITU consultations on conformance
assessment and interoperability are a useful
discussion platform, see e.g.
• Presentations at Kenya workshop, in particular
 TSB’s overview presentation on status of
implementation of WTSA-08 Res. 76
6
Next Steps/Actions
 Continue implementing steps outlined under
“Strategic directions”
 Study groups to continue working on test specs
 Keep momentum of ITU’s IPTV Interop event
• Expand to other ITU standards, in collaboration with
partners as appropriate
 Populate conformance database
7
Supplementary Slides
8
ITU-T SG 11’s Questions on Testing
 ITU-T SG 11: Signalling, protocols and test
specs
• Q 8/11 Protocol Test Specifications for NGN
• Q 9/11 Monitoring parameters for NGN
protocols
• Q 10/11 Service test specification for NGN
• Q 11/11 QoS tests specification for NGN
• Q 12/11 NID and USN test specification
ITU-T SG 11 Approved Standards
 Q.3900 Methods of testing and model network architecture for NGN
technical means testing as applied to public telecommunication
networks
 Q.3901 Distribution of tests and services for NGN technical means
testing in the model and operator networks
 Q.3902 Parameters to be monitored in the process of operation when
implementing NGN technical means in public telecommunication
networks
 Q.3903 Formalized presentation of testing results
 Q.3904 The scenarios, list and types of tests for TM local and NUT
testing for IMS on the model networks
 Q.3910 Monitoring parameters set for NGN protocols
 Q.3911 Monitoring parameters set for voice services in NGN
10
ITU-T SG 15’s Extensive Testing
Ecosystem
 ITU-T SG 15: “Optical transport networks and
access network infrastructures”
 Tight collaboration:
• Access (DSL, PON):
 Broadband Forum
• Optical transport, framing, cross connecting
 Optical Internetworking Forum (OIF)
 Metro Ethernet Forum (MEF)
• Home networking
 HomeGrid Forum
 Broadband Forum
Nairobi, Kenya, 30-31 July 2010
11
ITU-T SG 17’s Testing languages,
methodologies and framework
 Develop conformance testing methodology and
framework
 Develop interoperability testing methodology and
framework
 Assist standards developers for applying
conformance and interoperability testing
methodology to their specific needs
 Progress work in the area of Testing and Test
Control Notation, version 3 (TTCN-3)
ITU-T SG 17’s approved standards
Conformance Testing
Methodology and Framework









X.290 - General Concepts
X.291 - Abstract Test Suite Specification
X.292 - Tree and Tabular Combined
Notation
X.293 - Test Realization
X.294 - Requirements on Test
Laboratories and Clients
X.295 - Protocol Profile Test
Specification
X.296 - Implementation Conformance
Statements
X Suppl. 4 - Supplement on generic
approach to interoperability testing
X Suppl. 5 - Supplement on
interoperability testing framework and
methodology
Testing and Test Control
Notation version 3 (TTCN-3)










Z.161: TTCN-3 core language
Z.162: TTCN-3 tabular presentation
format (TFT)
Z.163: TTCN-3 graphical presentation
format (GFT)
Z.164: TTCN-3 operational semantics
Z.165: TTCN-3 runtime interface (TRI)
Z.166: TTCN-3 control interface (TCI)
Z.167: TTCN-3 mapping from ASN.1
Z.168: TTCN-3 mapping from CORBA
IDL
Z.169: TTCN-3 mapping from XML
data definition
Z.170: TTCN-3 documentation
comment specification
ITU-T Recs &
test suites
Supplier’s
conformity
route
decision
Test results
(ITU-T X.290)
Accredited
(ISO/IEC Guide 65)
Certification body
Evaluation
1st party
Evaluation
Conformity
Certificate issued by
Certification
Body
Supplier’s Conformity
Declaration
(ISO/IEC 17050)
ITU CIP services
Supplier’s
Request to ITU
ITU Conformity
Database
Implementation
of the ITU Conformity
Test results
(ITU-T X.290)
Test lab
(certification body
responsibility)
Programme
Conformity Assessment / Certification
14
1st 2nd or 3rd party
accredited lab
(ISO/IEC 17025)
Supplier’s Declaration of Conformity – SDoC 1/2
15
Supplier’s Declaration of Conformity – SDoC- 2/2
16
ITU’s 1st Interop Event (1/2)
 Putting to the test:
ITU-T’s IPTV suite of standards: H.721 et al.
 Geneva, 20-23 July: testing, followed by showcasing
 Participating companies highly satisfied: 4.5 rating (out of 5)
 Event highlight on YouTube, ITU website:
http://youtube.com/watch?v=CpJmI_5aQrA
 ITU press release: http://itu.int/net/pressoffice/press_releases/2010/31.aspx
 Photos on Flickr: http://flickr.com/photos/itupictures
ITU’s 1st Interop Event (2/2)
Test report will be fed into the standards development process
H.721 Conformity Check List
H.770 Conformity Check List
Testing specs:
• HSTP.CONF-H721
• HSTP.CONF-H762
• HSTP.CONF-H770
H.762 Conformity Check List
H.750 Conformity Check List