10/31/2011
ITU Forum on Conformance and Interoperability Testing in
CIS and Europe Regions
(Moscow, Russia, 9-11 November 2011 )
TESTING EXPERIENCE OF
A1 - TELEKOM AUSTRIA
Martin Brand
(WP 4/11 Vice-chairman)
Moscow, Russia, 9-11 November 2011
TESTING EXPERIENCE OF A1 TELEKOM AUSTRIA
ETSI TC INT Vice Chairman
1
10/31/2011
Presentation Outline
• Used NGN Testing methods
−
−
−
−
−
−
Tested Interfaces
Interfaces which can be tested with conformance tests
Limitations of the strategy
Testing experiences
Tested Interconnection Interfaces
TTCN-3 test systems, a system with limits ?
• QoS
– QoS Testing experiences
• Conclusions
Testing experience of A1 Telekom Austria
3
Used NGN testing
methods
Testing experience of A1 Telekom Austria
4
2
10/31/2011
IOT with Conformance Checking
Testing experience of A1 Telekom Austria
5
Characteristics and limitations of
Conformance Testing
• Characteristics
– Gives a high-level of confidence that the
standardised parts of a product are working as
specified
– It is component (Black Box) testing
– Requires a test system (i.e., executable test cases)
– High degree of control and observation
– Tests are thorough and accurate but limited in scope
• Limitations
– Does not necessarily prove interoperability with
other products
– Tests are focussed on part of a product
– Test systems may be expensive
Testing experience of A1 Telekom Austria
6
3
10/31/2011
Used Conformance and Interoperability Testing configurations
TTCN-3
TTCN-3
A
d
i g i
B
t a l
d i g i
t a l
1 2
6
9
8
#
*
Conformance testing
(of terminal equipment)
Conformance testing
(of a network element)
1 2 3
1 2 3
4 5 6
4 5 6
7 8 9
7 8 9
*
3
4 5
7 8
8 #
*
8 #
Interoperability testing
(of terninal equipment)
TTCN-3
TTCN-3
Testing experience of A1 Telekom Austria
7
Interfaces which can be tested with
conformance tests
IP Multimedia Networks
Legacy mobile
signalling Networks
CS Network
Mb
Mb
Mm Ma
CS
Mk
Sh
Mg
LRF
C, D,
Gc, Gr
Cx
Mk
Mw
Mj
Cx
Dx
MGCF
Mg
Mn
S-CSCF
ISC
MRB
Mw
Cr
SLF
Dx
Mr
Mb
Mm
HSS
BGCF
Mi
IMSMGW
Le
AS
I-CSCF
BGCF
CS
LCS
Client
Ml
Dh
E-CSCF
Rc
ISC
Mw
MRFP
MRFC
EATF
Mx
Mp
Mb
Mb
Mb
Mx
Mx
Rx
IBCF
Gm
BGCF
Mx
Ici, Mm
Ix
TrGW
UE
Mi
I4
P-CSCF
Izi
Ut
Testing experience of A1 Telekom Austria
8
4
10/31/2011
Limitations of the strategy
• Due to the limited number of tested interfaces
(e.g. no vertical interfaces), the full system
conformity of all elements can not be ensured
• As a consequence the system components which
interfaces can not be tested (e.g. H.248, RACS
Interfaces)should be used from one vendor to
ensure the IOP between the different network
components
• To ensure the end-to-end functionality, internal
E2E tests with different interfaces must be
executed
• As an option, tests performed with tools supplied
from the vendors can be executed
Testing experience of A1 Telekom Austria
9
Testing experiences
• The conformity of the products from
some vendors could be not supported.
• These products are mostly cheaper,
because they are not tested according
to the standards
• The time for “go in service” will be for
such products much longer
Testing experience of A1 Telekom Austria
10
5
10/31/2011
Tested Interconnection Interfaces
24.229 Mx Interface
24.229 Mx Interface
24.229 Mx Interface
Testing experience of A1 Telekom Austria
11
TTCN-3 test systems, a systems
with limits ?
• Characteristics
– TTCN-3 specifies the test structure but a specific system is needed for the
execution
– The test system architecture is defined with the TRI (TTCN-3 Runtime
Interface) and TCI (TTCN-3 Control Interface) standards
– TTCN-3 allows reuse of test platforms with different tools but also for different
SUTs
– A test system requires
• A TTCN-3 tool = TTCN-3 compiler and execution environment
• A test platform for a specific device under test
• Limitations
- The SUT and platform adapter (TRI interface) is proprietary
- The production of the TRI adapter is expensive ( 20 k€ - 30 k€) per ATS
- With each release change of the TTCN-3 Compiler the platform adapters must
be adapted
- If the test equipment is not supported any more from the company which is
developed the adapters the ATS can not be used with new releases
- limited number of tests platforms
- for the ATS validation are three SUT needed which causes a delay from 1-3 Year
until a validated ATS is on the market
Testing experience of A1 Telekom Austria
12
6
10/31/2011
TTCN3 Tests System and Interfaces
Test System Executor
TTCN-3 Test System
Test Control
Logging
Codecs
TCI
[compiled]
TTCN-3 Executable
TRI
SUT Adapter
Platform Adapter
System Under Test (SUT)
TCI = TTCN-3 Control Interface
TRI = TTCN-3 Runtime Interface
Testing experience of A1 Telekom Austria
13
Protocol Test Equipment
Tester used by A1 Telekom Austria
• CATAPULT & Testingtech TTCN-3 compiler
• NAVTEL
• Tektronic K1297
• Tektronic Spectra 2
• ARCATECH Harmony
• Empirix Hammer
Testing experience of A1 Telekom Austria
14
7
10/31/2011
Telekom Austria
Voice QoS Test Configuration
Malden
GL Communication
Head Acoustics
IXIA
Testing experience of A1 Telekom Austria
15
Head acoustic
• ACQUA is a dual channel analysis system for diagnosis of
acoustic and/or electric transmission paths up to 24 kHz.
• Applications
– Quality control Conformance tests
– Speech quality measurement and optimization (algorithms and systems)
– Speech quality evaluations of - standard terminals (analog, digital) mobile phones
(UMTS, GSM, CDMA, TDMA ...)
– cordless phones (DECT, CAT-iq™ ...)
– hands-free and conference terminals (office, vehicle ...)
– multimedia terminals (VoIP-Systems ...)
– network components (echo canceller, DCME, speech codecs ...)
Testing experience of A1 Telekom Austria
16
8
10/31/2011
Malden
• Terminal development
• SIP phone test bed
• UMA performance and
handover analysis
• Enterprise SLA
validation
• Echo canceller
analysis
• Jitter Buffer Tests
• Train communications
• DTMF Analysis
• Regression testing
• Conference bridge
testing
• Speech quality
optimisation in all
networks
• Speech quality /
load evaluation
• Drive Test
• Codec evaluation
• Electro-Acoustic
conformance
• Network element
development
• Wireless handset
comparison
• Vendor selection
• Echo simulation
• Competitive analysis
of cellular networks
• DSP performance
measurement
Testing experience of A1 Telekom Austria
17
IXIA
• Key Features
– ITU-T T.38 support
– Predefined test scenarios for automating functionality,
interoperability, and conformance testing
– Protocol flow functions shaped as visual blocks, no
programming or other script language is needed.
– Simulate multiple IAFDs and fax gateways
– T.30/T.38 Fax Protocol Analysis
– T.30 signal trace extracted from T.38 packet trace
Testing experience of A1 Telekom Austria
18
9
10/31/2011
GL Communication
Testing experience of A1 Telekom Austria
19
QoS Testing experiences
The user applications (fax & modem)
are mostly distorted due to:
− end-to-end delay bigger than 100 ms
− problems with not correct implemented
Echo Cancellers and Jitter Buffers in the GW
− not interoperable H.248 and SDP procedures
within the network
− T.38 is not sensible on the delay, but during
the TDM- IP Transit Scenario with more
TDM – IP segments T.38 does not work.
− Not time synchronized network elements
Testing experience of A1 Telekom Austria
20
10
10/31/2011
Conclusion
• The conformity of the products from some vendors could be
not supported. The time for “go in service” will be for such
products much longer.
• The fax & modem transmissions are mostly distorted due to
end-to-end delay bigger than 100 ms and problems with not
correct implemented Echo Cancellers and Jitter Buffers
• Not interoperable H.248 and SDP procedures within the
network
• The testing of the NGN is mostly complicated caused due to
different interfaces
• To ensure the end-to-end functionality, a combination of
conformance tests and internal E2E tests with different
interfaces must be executed
Testing experience of A1 Telekom Austria
21
THANK YOU!
Martin Brand
martin.brand@A1telekom.at
11