BENEFITS OF CONFORMANCE AND INTEROPERABILITY TESTING
PERSPECTIVES FOR 2012
Francois Fischer, Project Manager, ERTICO
Sebastian Mϋller, Technical Coordinator, ETSI
Why Produce Test Specifications?
A proven way to ensure interoperability and
standard compliance of ITS implementations
Their development provides a first validation
of the base standard(s)
Leads to significant improvements of base
specifications
Provide a basis for certification of products or
a test plan for an interoperability event
2
BPI: Acheiving Interoperability
Types of Testing
Conformance testing
• Testing a single part of a product for its compliance to a
base standard
• See Demo of Conformance Validation Framework
Interoperability testing
• Checking end-to-end functionality between a collection of
equipment or devices
Interoperability testing with conformance checking
• Testing interoperability while monitoring conformance
•
3
parameters
This was applied for 1. ITS CMS Plugtest
BPI: Acheiving Interoperability
What is a Plugtests event?
A test event organized and run by a neutral body
• Scope, test infrastructure and test scenarios based on standard
• Scheduling
• Test Results and Feedback to Standards Development
An opportunity for engineers
• Evaluate the interoperability of their products
• Validate their understanding of the base specification
• Save time
An opportunity for vendors
• To demonstrate end 2 end interoperability to operators/end customers
• Promote the technology and community
An opportunity for Standards Development
• Gaps, ambiguities, interpretations
• A tool to validate and enhance the quality of standards
4
Interoperability Test Procedure
Connect two or three devices from different vendors
Check connectivity between devices
Perform tests according to Plugtest Guide
• Check if devices can send/receive frames from each other
• Check if data is handled correctly in the network and facility layers
• Check if implemented algorithms work correctly
Result determination and reporting
• Result OK: run next test
• Result NOK: check monitor tools to identify source of error
• Report results in ETSI Test Reporting Tool
5
First ITS CMS Plugtest
Jointly organized by ETSI and ERTICO
Hosted by TNO
Sponsored by EC
Test specification produced by ETSI
• Distributed 2 months prior to event
• Total of 33 tests
• Developed in cooperation with DRIVE-C2X and EcoMove
ETSI – CTI Tools
• WIKI
• Scheduling Tool
• Test Reporting Tool
6
Participants
# Implementations
1
Cohda /NXP
2
CTAG
3
DENSO
4
Hitachi
5
Honda
6
ITRI
7
NEC
8
NordSys
9
Peek /TNO
10
QUWIC
11
Q-Free
12
Siemens
13
Vector
14
VTT
7
# Tools
1
Fokus
2
TNO
3
TestingTech
4
Vector
Pictures
Test
session
Test planning
Wiki
Debriefing
Results
8
© 2010 – Proprietary Information of ETSI
Benefit for DRIVE-C2X
DRIVE-C2X designed a “Reference System” ITS station based on the ETSI TC
ITS standards:
• The “Reference System” will be then integrated in Vehicle or Roadside ITS
•
stations by different project partners
DRIVE-C2X partners performed integration tests with “Reference System”.
DRIVE-C2X specific challenges:
• SW components developed by different partners, then ported to different HW
platforms and deployed on roadside infrastructure
The interoperability tests performed during the Plugtests event made the
project partners confident on the quality of the “Reference System”:
• Successful test sessions executed between DRIVE-C2X partners – minor issues
•
•
9
were solved
Test results confirmed that the DRIVE-C2X “Reference System” is compatible
with other vendor implementations as well as the EcoMove project
implementations
The results prove the good quality of the reference ETSI TC ITS base standards
© ETSI 2011. All rights reserved
The ITS interoperability
workshop
Aimed to disseminate the experience and lessons learned
during the Plugtests
Presentations from:
• DRIVE-C2X, ERTICO, ETSI, Fraunhofer FOKUS, Renault, SISCOGA,
TNO, VECTOR
Round table about the “future steps”:
• With the participation of Honda, InnovITS, Renault and TNO
• Why deployment requires Interoperability
• How Plugtests events can accelerate standardization
• Conformance testing and Plugtests event to pave the way for
certification
TNO demo on cooperative ITS network emulator
10
© ETSI 2011. All rights reserved
Pictures
2nd ITS CMS Plugtests event
11 – 15 June 2012
Hosted at IFSTTAR, Versailles, France
• http://www.ifsttar.fr/presentation/implantations/versailles-satory/
Scope
• optimized CAM, DENM
• SAM
• TOPO, SPAT
• All optional tests from 1st ITS CMS Plugtest
All facilities for lab test are available
• Faraday cages
• Test Network
• Central Position Server for GPS feeds
Anybody interested in this event please contact
sebastian.mueller@etsi.org and f.fischer@mail.ertico.com
12
Links
The Plugtests event:
• http://www.ertico.com/1st-interop-event-cms-14-18nov-2011
• http://www.etsi.org/plugtests/ITS_CMS/Home.htm
The Workshop
• http://www.ertico.com/1st-interop-event-workshop/
• http://www.etsi.org/plugtests/ITS_CMS/Workshop.htm
13
© ETSI 2011. All rights reserved
Other ITS domains
Plugtests event are planned for other ITS
technologies:
• eCall
• A Plugtests event is planned 21-25 May 2012 at
InnovITS ADVANCE at Nuneaton (Birmingham, UK)
• TPEG
• A plugtests event is planned Q3 2012 at ERTICO in
Brussels
14
© ETSI 2011. All rights reserved
22 - 26 October 2012