ITU Regional Seminar for the Africa Region on Conformance
and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
ITU-T standardization activity on testing.
Global approach for C&I Testing on the Model
Networks
Denis Andreev
Director of Technopark ZNIIS,
Rapporteur of Q.10/11 SG11 ITU-T
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
Contents
ITU-T activities in testing area
Testing methodology history
Technical means (equipment)
interoperability
Model Networks
The services interoperability
The QoS interoperability
Benchmarking
Conclusions
2/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
1
ITU-T activities on testing area (1)
Resolution 76 “Studies related to
conformance and interoperability
testing, assistance to developing
countries, and a possible future ITU
mark program” was approved at the
World Telecommunication Standard
Assembly (Johannesburg, 2008)
3/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
ITU-T activities on testing area (2)
“The interoperability of international
telecommunication networks was
the main reason to create ITU in the
year 1865 (International Telegraph
Union), and that this remains one of
the main goals in the ITU strategic
plan”
4/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
2
What is the
interoperability today?
5/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
ITU-T activities on testing area (3)
Recommendation Y.101 “Global
Information Infrastructure
terminology: Terms and definitions”
Interoperability is the ability of two
or more systems or applications to
exchange information and to mutually
use the information that has been
exchanged”. This Recommendation
was approved at March 2000. It’s so
long ago from today point of view.
6/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
3
The testing methodology history (1)
1.
2.
1995, ITU-T (X.290), ETSI (ETS
300406) – conformance testing
based on the ISO/IEC 9646 and
specific telecommunication criteria
and features.
1999, ETSI (TR 101667) – Network
Integral (Interconnection) testing
(end-to-end, node-to-node).
7/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
The testing methodology history (2)
The conformance testing goal is the
specification profiles verification. The network
testing goal is the correct, integrity and
reliability services for users.
The table 1 ETR 101 667 defines the users of
both methodologies. The conformance testing
methodology users are the vendors at first
place and Administrations and operators
optionally, the network testing methodology
users are the Administrations and operators
only
8/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
4
The testing methodology history (3)
The methodology of the model network
using for network testing was proposed
at ITU-T in 2004 (Q.39xx series)
The model network is the network,
which simulates the capabilities similar
to those available in present
telecommunication networks, has a
similar architecture and functionality
and users, the same telecommunication
technical means
9/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
The NGN testing experience
ZNIIS Model Network (2004-2011,
SSW, IMS, BBA) 6973 tests – 7 %
unsuccessful tests
Plug Test ETSI (Slovenia, 2008, IMS)
410 tests – 18% unsuccessful tests
(Joint ITU-T/ETSI meeting, Moscow,
ZNIIS, 10 April, 2009)
10/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
5
Last approval the Q.39xx
Recommendation list
Q.3900
Q.3901
Model Network Architecture (08)
NGN testing specification (08)
Q.3903
Q.3904
Knowledge Data Base (08)
NGN (IMS testing specification) (10/10)
Q.3906.1
Broadband access testing specification
(wired) (06/10)
Q.3909
An overview of NGN Interoperability testing
methods (draft)
Q.3910
Monitoring parameters set for NGN
protocols (06/10)
Monitoring parameters set for voice services
in NGN (06/10)
Q.3911
Handbook on testing (09/11)
11/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
ITU-T Rec. Q.3900 as a global
approach of testing
Purpose – determination of the Model network as
a basic solution for NGN testing
Model network – a network which simulates the
capabilities similar to those available in present
telecommunication networks, has a similar architecture
and
functionality
and
uses
the
same
telecommunication equipment
Contents of Q.3900
Classification of NGN TM regarding the NGN
functionality (Y.2012)
Determination of testing procedures
Requirements to the Model network
12/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
6
Basic Methodology of NGN testing.
ITU-T Recommendation Q.3900 (09/2006)
13/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
Classification of NGN Technical
Means to be tested
- Call Session Control System
-Media Gateway Controller (MGC)
-Proxy Server SIP (PS)
-IP Multimedia Subsystem (IMS)
- Voice and signaling transmit
system
-Media Gateway (GW)
-Signaling Gateway (SG)
-Transport Network Environment
(TNE)
- Application servers
-Application Server (AS)
-Media server (MS)
-Messaging Server (MeS)
- Management and billing
system
- Management System (MS)
- Billing system (BS)
- Access Environment
- NGN Integrated Access
Devices (NGN-IAD)
- Media gateway for Legacy
Terminal Equipment (GW-LTE)
14/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
7
NGN functional architecture
Recommendation of ITU-T Y.2012
A p p lic a tio n s
Management functions
( m a y in c lu d e o w n A u th e n tic atio n , A u th o riza tio n a n d A cc o u n tin g )
S e rvic e
c o n tro l
S -5 : S . U se r
P r o f ile F E
S - 7 : In te rc o n n e c tio n
B o r d er G a tew a y
C o n tro l F E
S -1 5 : G e n e r a l
S e r v ic e s C o n tr o l F E
S - 2 : P r o x y C a ll S e ssio n
C o n tr o l F E
S - 1 : S er v in g
C a ll S e ssio n C o n tr o l F E
S -8 : A cc e ss G W
C o n tr o l F E
S - 1 3 : M e d ia R e so u r ce
C o n t ro l F E
L e g a cy
T er m in a l
T - 1 6 : P o lic y D ec isio n F E
T -1 0 :
N e tw o r k A c c e s s
C o n fig ur a tio n F E
RGW
T - 4 : A cc e ss
R e la y F E
C o re tra n sp o rt
T -9 :
S ig n a llin g
G a te w a y
FE
T -8 :
M e d ia
R e so u rc e
P ro ce ss in g
FE
T -6 :
In te r co n n e c tio n
B o r d er
G a tew a y F E
A c ce ss tra n sp o rt
T -3 : E d g e
N ode FE
T -2 : A c ce ss
Node FE
T -1 : A c ce ss
M e d ia
G a tew a y
FE
IP
M u ltim e d ia
N e tw o rk s
T -1 7 : T r a n sp o r t
R e so u r ce C o n tr o l F E
T -1 7 : T r a n s p o r t
R e so u rc e C o n tro l F E
T -1 4 : A c c e s s M a na g e m e n t F E
C u st o m e r
N e tw o r k
O th e r N G N
T -1 6 : P o lic y D ec isio n F E
T -1 3 : T . L o c a tio n
m a n ag em e n t F E
T -1 1 :
T . A ut h e n tic a tio n
& A u th o r iza t io n
FE
S -1 0 : B re a ko u t
G a tew a y C o n tr o l
FE
S -9 : M e d ia
GW
C o n t ro l F E
RACF
RACF
T -1 5 : H o m e G a te W a y
C o n fig ur a t io n F E
T -1 2 : T . U s e r
P r o file F E
L eg a c y
T e r m in a l
S -1 2 : N e tw o r k
S ig n a lin g
In te rw o r kin g F E
S -3 : In ter ro g a t in g C a ll
S e ssio n C o n tr o l F E
S -4 : S u b sc r ip tio n
L ocator F E
NACF
NGN
T er m in a l
S -6 : S .
A u th e n tic a t io n &
A u t h o r iz a t io n F E
S - 1 4 : M e d ia
R e so u rc e B r o ke r F E
Application
A p p lic a tio n S u p p o r t F u n c tio n s & S er v ic e S u p p o rt F u n ctio n s
S -1 1 : U se r S ig n a lin g
I n te r w o r kin g F E
NGN
E n d -U se r U N I
F u n c tio n
ANI
*
T -5 :
A c ce ss
B o r d er
G a te w a y
FE
A c c e ss P a c k e t T ra n sp o rt F u n c tio n s
C o re P a c k e t T ra n sp o rt F u n c tio n s
T -7 :
T r u n ki n g
M e d ia
G a tew a y
FE
Scope of N G N
P S T N /IS D N
15/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
Conformance of NGN Functions to NGN
Technical Means to be tested
NGN Technical means
NGN Functionality
Call Session Control System
Media Gateway Controller (MGC)
S3, S7, S9, S10, S12
Proxy Server SIP (PS)
S2, S3, S7, S11, S12
T10, T11, T12, T13
T10, T11, T12, T13
IP Multimedia Subsystem (IMS)
S1, S3, S6, S7, S8, S10, S12, S13
T10, T11, T12, T13, T14, T15, T16, T17
Voice and signaling transmit system
Media Gateway (GW)
T7, T8
Signaling Gateway (SG)
T8, T9
Transport Network Environment (TNE)
T5, T6, T8
Application servers
Application Server (AS)
S4, S5, S6, S14, S15
Media server (MS)
S4, S5, S6, S14, S15
Messaging Server (MeS)
S4, S5, S6, S14, S15
Management and billing system
Management System (MS)
Billing system (BS)
-
error processing management
equipment configuration management
billing system management
service management
security management
Access Environment
NGN Integrated Access Devices (NGN-AD)
Media gateway for Legacy Terminal Equipment
(GW-LTE)
T2, T4, T3, T5, T15, T14
T1, T2, T3, T4, T5
16/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
8
Architecture of model network
17/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
ZNIIS Technopark Model network
Rec. ITU-T Q.3900 Methods of testing and model network architecture for NGN technical
means testing as applied to public telecommunication networks
18/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
9
The services interoperability testing (1)
The ITU –T recommendations on the
services interoperability testing are
absent today. Furthermore, the
service scenarios for important NGN
services are absent too. It’s very
complicated problem today, but
during ITU-T study period 2009-2012
the key recommendations set for
service interoperability testing
should be developed
19/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
The services interoperability testing (2)
The detailed requirements for the services
implementation should be developed. It could
be include:
service definition and features
network capabilities
network architecture and function elements
access network types and user equipments
service delivery scenarios
call flows
reference points and protocols
service implementation on the non-IMS
network
interworking with others services
20/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
10
The services interoperability testing (3)
The testing scenarios, list and types
of tests for NGN TS1 basic call and
supplementary services, NGN TS1
streaming services and NGN TS1
multimedia services should be
developed as separately ITU-T
Recommendations
21/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
The Q.39xx recommendation list
(services testing)
Q.3945 – TS1 (will be approve in Sept.
2011)
Q.3946 – Testing scenarios, list and types
of tests for NGN (TS1) basic call and
supplementary services
Q.3947 - Testing scenarios, list and types
of tests for streaming services (TS1)
22/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
11
The QoS interoperability aspects
The many networks are realized NGN
concept. At first, of course, it’s the
basic IP network. Furthermore, it could
be WiFi (based on Ethernet
technology), IPTV and so on
23/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
The QoS interoperability aspects
The some QoS mechanisms were
developed for QoS supporting at IPbased networks. The recommendation
Y.1291 which was approved in 2004
determine four standardized
approaches for QoS supporting:
Integrated services (IntServ),
Differentiated services (DiffServ),
MultiProtocol Label Switching (MPLS),
IPCablecom Dynamic QoS
24/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
12
The exist problematic on QoS
support
The network deployment
switching technologies
based
on
packet
Service distribution without correspondence with
telecommunication technologies
The payload increase in transport network in
consequence of increase broadband of access
layer (on 50 % each of the year) Jakob Nielsen
Heterogeneous network environment (TDM and
IP)
Reduce of reliability rate
different vendors solutions
in
case
of
usages
25/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
The typical approach and common
problems on QoS astimation
At the moment the three basic approach are widely
used for estimation quality of distributed mediainformation contents on the exist operators
networks (Rec. ITU-T G.1011):
•active (intrusive, PESQ, POLQA)
•passive (non-intrusive, P.563)
•modeling (E-model)
26/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
13
The common problems
Active (intrusive, PESQ, POLQA)
the auxiliary test service traffic QoS estimation on
the exist network does not give the objective
situation on speech quality on the hole network
Passive (non-intrusive, P.563)
Comparison with the standardized model of the
traffic/service (the various services and their profiles
complicates process of control – model
standardization is required)
E-model
The mathematical model having a divergence with
real technical telecom solutions, applied on an
operator network
27/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
INNOVATION APPROACH ON
ESTIMATION AND SERVICE QUALITY
CONTROL
Measurement and finding the limit values
for Network performance and network
productivity for each network segment for
distributing service sessions (Model
network – common instrument for this
task)
Implementation probes on real networks
which can simulate services like distributed
on exist operators network and which can
monitoring/control quality of service
28/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
14
THE ARCHITECTURE OF MODEL NETWORK FOR
NETWORK PERFORMANCE, QOS AND
NETWORK PRODUCTIVITY TESTING
29/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
Substantiation of innovation approach
necessity (joint project ITU-ZNIIS CIS-08)
Ethernet-Ethernet
MOS 5
Service
G.711
512 кбит/с
4,41 4,41
4,41
3,72
3,7
4
3,71
2,79
3
Session 14
Канад
1,71
2
Session 23
Канал
1
Канал 1
1
1
0
Канал 2
Session 32
Канал
Session 41
Канал
Канал 3
1
Канад 4
2
3
4
Simultaneously sessions mode
LNCS. Springer 2011 “Limited values of network performance and network productivity
estimation approach for services with required QoS. Service Benchmarking“
Denis.Andreev, Konstantin.Savin, Victor.Shalaginov, Viya.Zharikova, Sergey.Ilin
30/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
15
Substantiation of innovation approach
necessity (joint project ITU-ZNIIS CIS-08)
Service
G.729
512 кбит/с
Ethernet-DSL
MOS
Канал
99
Session
5
Session
Канал
88
Session
Канал
77
4
Session
Канал
66
3
Session
Канал
55
Канал 1
Канал 2
Канал 3
Канал 4
Канал 5
Канал 6
Канал 7
Канал 8
Канал 9
2
1
0
7
8
Session
Канал
44
Канал
3
Session
3
Session
Канал
22
Session
Канал
11
9
Simultaneously sessions mode
31/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
Key tasks for estimation QoS on the Model
networks
The norms on QoS/NP on the network
segment determination for different
telecom technologies and services
Service modeling
Benchmarking solutions and networks
Interoperability testing
Testing on RFP conformance
Determination of optimal functionality of
QoS support system before implementation
on network
Practical training courses for increase
network operator’s personal qualification
32/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
16
Benchmarking Recommendations list (1)
Q. Bench–IMS–NGN-A
Q. Bench–IMS–NGN-B
Q. Bench–IMS–NGN-C
Q. Perf-Bench-IMS
IMS/NGN Performance
Benchmark Part 1:
Core Concepts
IMS/NGN Performance
Benchmark Part 2:
Subsystem Configurations
and Benchmarks
IMS/NGN Performance
Benchmark Part 3:
Traffic Sets and Traffic
Profiles
IMS/NGN Performance
Benchmark Part 4:
Delay objectives for
various IMS transactions
33/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
Benchmarking Recommendations list (2)
Q. Bench-PES NGN A
PES Performance
Benchmark Part 1:
Core Concepts
Q. Bench-PES NGN B
PES Performance
Benchmark Part 2:
Subsystem
Configurations and
Benchmarks
Q. Bench-PES NGN C
PES Performance
Benchmark Part 3:
Traffic Sets and Traffic
Profiles
34/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
17
Conclusions
1.
The main goal of testing today should be
network testing (interoperability) for
supporting the correct, integrity and
reliability services for users.
2.
This goal could be supported by Model
network methodology.
3.
The interoperability issues include the
technical means, services, QoS classes
and parameters. All of these features create
the Global Interoperability, which could be
tested on the Model network in complex.
35/36
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
ITU Regional Seminar for the Africa Region on Conformance
and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
Denis Andreev
Director of Technopark ZNIIS,
Rapporteur of Q.10/11 SG11 ITU-T
tel: +7-495-368-8745
mobile: +7-495-647-9603
fax: +7-495-368-9105
skype: davwilly77
sipnet: 2811971@sipnet.ru
E-mail: andreevd@zniis.ru
cc: andreevd@ties.itu.int
_____________________
Russia, 111141, Moscow,
1-st Proezd Perova polya, 8
ITU Regional Seminar for the Africa Region on Conformance and Interoperability Testing Centre(s)
Accra (Ghana), 4-6 July 2011
18