ITU Workshop on “Monitoring and Benchmarking
of QoS and QoE of Multimedia Services in
Mobile Networks”
(Buenos Aires, Argentina, 24-25 July 2014)
Update on ITU-T & ETSI standardization
activities for LTE QoS / QoE
Dr. Peter Počta
Peter.Pocta@fel.uniza.sk
University of Žilina, Slovakia
Joachim Pomy
Consultant@joachimpomy.de
OPTICOM, Germany
Version : 11 December 2008
 ITU-T SG 12, Question 11:
"Performance interworking and traffic management
for Next Generation Networks "
Buenos Aires - Argentina - 24 - 25 Julyl 2014
2
G.VoLTE (1)
C-127: Inputs for G.VoLTE (Huawei Technologies Co.
Ltd., France Télécom Orange)
At last SG12 meeting, two contributions (C.074 from
Orange and C.076 from Ascom) addressed the need
for starting the development of specific
requirements on end-to-end Qos for voice over 4G
mobile networks. The decision has been taken to
develop a new Recommendation, provisionally
called G.VoLTE, under Q.11 (see the report R.4 of
WP3), with the following description:
Buenos Aires - Argentina - 24 - 25 Julyl 2014
3
G.VoLTE (2)
This Recommendation describes the key aspects
impacting end-to-end performance of managed
voice applications over LTE networks.
Buenos Aires - Argentina - 24 - 25 Julyl 2014
4
G.VoLTE (3)
This Recommendation makes assumptions about
relevant QoS mechanisms used to manage the voice
service, and studies the impact of such mechanisms
on the end-to-end network performance. Some
typical end-to-end scenarios are described, involving
cases with LTE access at both sides of the
communication, or with a different access
technology at one side (wireless or wireline access).
Buenos Aires - Argentina - 24 - 25 Julyl 2014
5
G.VoLTE (4)
A typical reference connection is defined in this
Recommendation, composed of various segments,
including: terminal, wireless access, backhaul
network, core network. Considerations regarding
the sharing of the budget of some key parameters
across these segments, such as latency, are
provided.
Buenos Aires - Argentina - 24 - 25 Julyl 2014
6
QoS of connections from current
technologies to LTE for delay sensitive applications
(ETSI STF 437)
Buenos Aires - Argentina - 24 - 25 Julyl 2014
7
Outline
Introduction of the ETSI STF 437 project (scope, team),
 QoS problems encountered by operators,
 Shortcomings of standards and implementations,
 Possible solutions (actions for standards and
implementations),
Buenos Aires - Argentina - 24 - 25 Julyl 2014
8
Introduction of the ETSI STF 437
 Project scope:
 address LTE related
QoS problems for
delay sensitive
applications,
 determine possible
shortcomings of the
standards and possible
shortcomings of
implementations
 propose possible
solutions
Buenos Aires - Argentina - 24 - 25 Julyl 2014
 Project team:
 Peter Počta (leader)
 University of Žilina,
Slovakia
 Jan Holub
 Mesaqin.com s.r.o.
(Ltd.),
Czech Republic
 Joachim Pomy
 Telecommunications &
Int’l Standards, Germany
 Catherine Quinquis
 France Telecom, France
9
QoS problems encountered by operators
 Contacted operators and stakeholders only trial LTE technology
 Trials mostly focus on a performance of data services (lab implementation).
 One of the contacted operators focused on voice service in its trial, following
problems have been encountered:
 Some problems with CS Fallback (was not working properly),
 Problem with routing between radio part (LTE network) and IMS,
 All problems have been fixed by vendor (wrong implementation).
 Conclusion: operators have not already started to deploy Voice over LTE in their
networks.
 To sum up current situation, according to market observers and visionaries:
 Network operators are planning to deploy VoLTE in upcoming years (2013 or 2014).
 The potential subscribers of LTE expect better performing data services than in 3G
and services that take advantage of the lower latency, such as voice and video.
(Quality of service – crucial role in an adaptation process of VoLTE).
 Implication: QoS should be properly implemented and tested in trials to ensure
sufficient quality level to speed up/drive the adaptation process of VoLTE.
Buenos Aires - Argentina - 24 - 25 Julyl 2014
10
Shortcomings of standards and implementations
 Shortcomings of standards:
 Standards focusing on IMS Multimedia Telephony,
IMS Profiles for voice service, SMS and
conversational video service:
 According to ETSI TS 126 114, the MTSI client
should only support AMR codec for voice service.
 The jitter-buffer management mechanisms of
video service described in the ETSI TS 126 114
seem to be very weak.
 The quality of experience metrics defined in clause
16 of ETSI TS 126 114 can be more or less only
considered as QoS parameters.
Buenos Aires - Argentina - 24 - 25 Julyl 2014
11
Shortcomings of standards and implementations
 Shortcomings of standards:
 Standards describing resource and admission control mechanisms
developed by 3GPP and ETSI (3GPP Policy and Charging Control (PCC)
and TISPAN Resource and Admission Control Subsystem (RACS)):
 3GPP PCC and TISPAN RACS vary considerably in architecture,
supported networks and node types (big functional differences).
 To harmonize them, a following activity called “Framework for Gq'
/ Rx harmonization” was started in 2007 by 3GPP.
 In 2009, the harmonization activity mentioned above has been
abandoned and all published specifications has been withdrawn
(the developed solutions has been lost).
 Main problem: no interface is available for communication
between the TISPAN RACS and the 3GPP PCC. Consequently, they
can not interact and negotiate with each other in order to finish
the operations required for continuous QoS guarantee (e.g.
resource reservation).
Buenos Aires - Argentina - 24 - 25 Julyl 2014
12
Shortcomings of standards and implementations
 Shortcomings of implementations:
 Interim solutions:
 Circuit-switched fallback (CSFB): Uses 2G/3G for voice and requires
interruption of the LTE connection; this is already affected by the
signalling of an incoming call.
 Voice over LTE Generic Access (VoLGA): Tunnels the voice call from the
2G/3G to the LTE terminal via an LTE data channel; requires additional
equipment in the network; lack of QoS.
 Fast Track VoLTE: Proprietary solution providing voice service via the LTE
technology; adds SIP stack to existing mobile network equipment.
 Over-the-Top solutions (OTT): VoIP applications operating via an LTE data
channel, running on the LTE terminal; lack of QoS.
 Migration of voice service to LTE:
 Before commercial deployment, a number of lab, field and market trials
should be conducted.
 User speech quality expectations are very high. If voice services do not
deliver the necessary level of quality and reliability, users will revert back
to existing circuit-switched options or, in some cases, simply rely on overthe-top solutions.
Buenos Aires - Argentina - 24 - 25 Julyl 2014
13
Shortcomings of standards and implementations
 Shortcomings of implementations:
 VoLTE Interop testing:
 was organised by the Multi-Service Forum (MSF) and backed by
the GSMA in September 2011 and October 2012.
 a couple of different aspects has been tested, mainly focusing on
signalling issues, multivendor interoperability.
 MSF reports in the white paper summarizing VoLTE Interop event
2011:
" The event proved that VoLTE is a viable solution for providing
voice services over LTE access technology, and can be deployed
with roaming and interconnect functionality to provide equivalent
service as CS-based voice services today.“
– What exactly has been tested?
– What kind of understanding of the terms "Quality of Service" and "service" have been
prevalent?
 After a detailed view into the scenarios of this interoperability
event, the STF 437 team has concluded that QoS of the media
channel is not tested at all.
Buenos Aires - Argentina - 24 - 25 Julyl 2014
14
Possible solutions (actions for standards and implementations)
 Actions for standards:
 Codec aspects:
 In principle, there are three possible solutions:
– Tandeming (lower quality and increased delay),
– End-to-end connection based on AMR codec (lower quality),
– using other codecs supported by IMS for end-to-end connections.
 STF 437 team advice: MTSI client defined in ETSI TS 126 114 should
have the possibility to access every codec supported by SIP.
 Action point: appropriate standard should be extended
accordingly.
 Jitter-buffer Management Mechanisms of video service:
– are currently not or at least not sufficiently standardized. Further
study in this area is of critical importance.
Buenos Aires - Argentina - 24 - 25 Julyl 2014
15
Possible solutions (actions for standards and implementations)
 Actions for standards:
 Quality of Experience metrics:
 There are still no clear standards in place, which can
support network operators in selecting those
parameters in the QoE extension.
 It is proposed by STF 437 team to follow the approach of
a Human Factors Extension to the Seven-Layer OSI
Reference Model proposed by Bauer and Patrick:
– Main idea: three additional HCI (Human Computer Interface) layers capture
the additional quality parameters introduced by the concept of QoE over QoS,
whereas the classical 7 OSI layers are synonymous to classical QoS parameters.
– The above mentioned three additional layers focusing on HCI are:
– Layer No.8: Display (keyboard, GUI/CLI, vocal, Bpp, Ppi, Ppm, etc.),
– Layer No.9: Human Performance (perception, cognition, memory, motor
control, social, etc.),
– Layer No.10: Human Needs (communication, education, acquisition,
security, entertainment, etc.) .
Buenos Aires - Argentina - 24 - 25 Julyl 2014
16
Possible solutions (actions for standards and implementations)
 Actions for standards:
 Framework for Gq’/Rx harmonization:
 Crucial to reopen the activity called “Framework for
Gq’/Rx harmonization”:
– define interactive interface supporting sufficient exchange of
information between TISPAN RACS and 3 GPP PCC
– harmonize the communication between the two different
resource and admission control solutions to satisfy the
subscribers’ service experience
Buenos Aires - Argentina - 24 - 25 Julyl 2014
17
Possible solutions (actions for standards and implementations)
 Actions for implementations:
 Migration of voice service to LTE:
 Delay values achieved in trials and real implementations have to be carefully
observed.
 Sufficient algorithms/techniques should be deployed in the networks in order to
ensure continuous data sessions for incoming and outgoing voice/video calls.
 A quality of service perceived by the end-user should be carefully measured in
trials and monitored in real implementations to ensure the best quality provided
to the users.
 VoLTE Interop testing:
 TC INT has begun to consider inclusion of QoS testing aspects into the
specifications for forthcoming VoLTE interoperability events.
 STF 437 team has proposed an example of improved test scenario presented on
next slide (Fig.1).
 While this is a good starting point, the challenge of QoS in heterogeneous
scenarios interconnecting current technologies with LTE may require additional
effort in the future.
 Another option: the organization of a pure Speech & Video Quality Test Event for
LTE.
Buenos Aires - Argentina - 24 - 25 Julyl 2014
18
Possible solutions (actions for standards and implementations)










LTE UE Attach (IP-CAN Session Establishment)
Tracking Area Update
LTE UE Detach (IP-CAN Session Tear Down)
IMS UA Registration (via LTE UE)
IMS Voice Session Establishment (LTE UE to LTE UE)
One-way delay, measurement in both directions
MOS-LQOx, measurement in both directions
Voice channel behavior under double talk and background noise conditions
IMS Voice Session Termination
MMTel Supplementary Service Interaction and Configuration
Figure 1: Example of improved test scenario for VoLTE interop test
Buenos Aires - Argentina - 24 - 25 Julyl 2014
19
TOWARDS QUALITY AND QOS ASSESSMENT FOR
VOLTE IN ETSI
ETSI TC INT
Chair: Giulio Maggiore, Telecom Italia
giulio.maggiore@telecomitalia.it
Buenos Aires - Argentina - 24 - 25 Julyl 2014
© ETSI 2013. All rights reserved
20
TC INT (Core Network and Interoperability Testing)
 Development of Core Network test specifications
(interoperability, conformance, performance, security) based
on 3GPP and E2NA specifications.
 To initiate and supervise interoperability events (such as
Plugtests) related to Core Networks as well as other events
(workshops and seminars)
 To coordinates interoperability efforts with other
organisations GSMA, OMA, IMS/NGN Forum, IMTC, etc.
 Endorsement of test specifications from/to other SDOs e.g.
ITU-T.
Buenos Aires - Argentina - 24 - 25 Julyl 2014
21
© ETSI 2013. All rights reserved
21
TC INT Interoperability Specification Work
R6
TS 186 011
V 1.1.1
2007
R1
R7
TS 186 011
V 2.1.1
Late R7
TS 186 011
V 2.2.1
2008
2009
R2
Late R2
Buenos Aires - Argentina - 24 - 25 Julyl 2014
22
R8
TS 186 011
V 3.1.1
TS 102 901
V 1.1.1
2010
R9
R10
TS 186 011
V 4.1.1
TS 102 901
V 2.1.1
TS 186 011
V 5.0.2
TS 102 901
V 5.0.6
TS 103 129
TS 103 129
V2.0.4
2011
2012 2013
E2NA
22
2013 – VOLTE QOS ASSESSMENT
TECHNOLOGY EVALUATION EVENT
ETSI STF#453
STF Leader: Peter Schmitting, FSCom
peter.schmitting@fscom.fr
Buenos Aires - Argentina - 24 - 25 Julyl 2014
23
© ETSI 2013. All rights reserved
23
VoLTE QoS Assessment event overview
 Scope: Technology evaluation Plugtests
aiming at validating the different
methodologies for end to end QoS
Assessment for VoLTE described in ETSI TS
103 189 over a deployed LTE/IMS network.
 Dates: 18-20 November 2013
 Venue: ETSI, Sophia Antipolis, France
 Web: http://www.etsi.org/news-events/events/698-volte-qosassessment-2013
Buenos Aires - Argentina - 24 - 25 Julyl 2014
24
© ETSI 2013. All rights reserved
24
VoLTE QoS Assessment event scope
 End to end tests performed with several different UEs over
a deployed LTE / IMS test network.
 4 Test Setups




Acoustic to acoustic
Acoustic to electrical
Electrical to acoustic
Electrical to electrical
 Voice quality tests based on instrumental assessment of
speech samples performed on different combinations of
the acoustic and electrical interfaces which allows
evaluating the different methodologies and setups.
 Parallel network performance evaluation
Buenos Aires - Argentina - 24 - 25 Julyl 2014
25
© ETSI 2013. All rights reserved
25
2013 – VoLTE QoS Assessment Technology
Evaluation event – test description TS 103 189
 End-to-end Tests based on Instrumental Assessment
of Speech Samples (POLQA model deployed):




Mean Opinion Score (MOS)-Listening Quality Objective
Delay variation over time
Difference in loudness level
MOS stability
 End-to-end Tests based on Instrumental Assessment
of Video Samples (PEVQ model deployed):






DMOS (Difference Mean Opinion Score)
Delay and Quality variation over time
Jerkiness and Blockiness
Blur, Chrominance and Luminance
Temporal Distortion
Frozen and Skipped Frames
Buenos Aires - Argentina - 24 - 25 Julyl 2014
26
2013 – VoLTE QoS Assessment Technology
Evaluation event – test description TS 103 189
 Other end-to-end tests in the voice channel:





End-to-end frequency response
Overall loudness rating
End-to-end delay
Quality of echo cancellation
Double talk performance
 End-to-end Tests based on functional parameters:
 Telephony Call Setup Time
 Streaming Service Access Time
 Network Performance Parameters
 UNI-to-UNI delay over time
 UNI-to-UNI jitter
 UNI-to-UNI packet loss
Buenos Aires - Argentina - 24 - 25 Julyl 2014
27
28
VoLTE QoS Assessment – Setup 1
Acoustic-to-Acoustic
Buenos Aires - Argentina - 24 - 25 Julyl 2014
© ETSI 2013. All rights reserved
28
VoLTE QoS Assessment – Setup 2
Acoustic to Electrical
Buenos Aires - Argentina - 24 - 25 Julyl 2014
29
© ETSI 2013. All rights reserved
29
VoLTE QoS Assessment – Setup 3
Electrical to Acoustic
Buenos Aires - Argentina - 24 - 25 Julyl 2014
30
© ETSI 2013. All rights reserved
30
VoLTE QoS Assessment – Setup 4
Electrical to Electrical
Buenos Aires - Argentina - 24 - 25 Julyl 2014
31
© ETSI 2013. All rights reserved
31
What’s next?
 Validation of 76% of the test cases in TS 103 189
 A number of gaps and potential improvements in TS 103 189
have been identified
 A new work item has been open for the update of TS 103 189
with the new setup, acoustic-to-electrical identified during
the event
 Next event will be launched when VoLTE Native Handset and
PEVQ tools will be available for testing
Buenos Aires - Argentina - 24 - 25 Julyl 2014
32
© ETSI 2013. All rights reserved
32
Further steps with regard to ETSI TS 103 189
 Verification of remaining voice quality test cases
tbd
 Verification of video quality test cases (RTPbased video streaming):
Test houses willing to do so are needed
 Extension of video quality test cases
towards MPEG-DASH
ITU-T SG12 Question 14 currently running a WI
called P.NATS
Buenos Aires - Argentina - 24 - 25 Julyl 2014
33
ITU-T SG12 Question14 Project P.NATS
 P.NATS focusing on a development of a model
 consisting of a collection of objective parametric quality assessment
modules
 predicting the impact of observed IP network impairments on quality
experienced by the end-user
 in multimedia mobile streaming and fixed network applications
 using progressive download, including adaptive streaming methods
 The development is divided into 2 parts, namely:
 Track No.1: bitstream-based parametric models (payload and/or header
information)
 Track No.2: full-reference and hybrid models for the video coding quality
module.
 Track No.2 is of our interest as it correlates well with an aim of VoLTE QoS
Test event
 Models for both tracks are supposed to be available at the end of this
year.
Buenos Aires - Argentina - 24 - 25 Julyl 2014
34
Preparation of new Project in EU context
 Test Methodologies for QoS, Mobile Broadband & Autonomic Networking
 Test methodologies for the interoperability with guarantied Quality of Services
(QoS) on the mobile broadband
 in a multi-vendor
 multi-network
 multiservice environment
 The test methodologies coming out from this project will be applied to the
preparation for the future ETSI Plugtests events.
 As holistic test methodologies, the proposal addresses also the test
methodologies for the future internet with
 autonomic networking,
 autonomic control and management.
 Such a global network infrastructure has the capabilities of
 self-establishing,
 self-configuring,
 self-optimizing and self-healing.
 For autonomics the decisions are made at the communication protocols,
functions, nodes and network-wide levels.
Buenos Aires - Argentina - 24 - 25 Julyl 2014
35
Any questions
?
Contact:
Consultant@joachimpomy.de
Buenos Aires - Argentina - 24 - 25 Julyl 2014
36