ITU Workshop on
“Workshop on Practical measurement of QoS/QoE
Parameters for Regulatory Compliance”
(Cotonou, Benin, 16-17 July 2012)
Country example of QoS measurement
of voice service
Case of Morocco
Hassan TALIB,
Vice-Chair ITU-T SG 12,
talib@anrt.ma // htalib@ties.itu.int
Cotonou, Benin, 16-17 July 2012
Presentation Scope
The QoS for Morocco telecommunications
A protocol for voice QoS measurements:
Choice of a subjective method,
List and definitions of indicators,
Measurements Methodology:
Definition of voice QoS measurement,
Sample and Statistic considerations,
Organization and Operation,
Process of Outcomes.
Recommendations to regulators/Op.
Cotonou, Benin, 16-17 July 2012
The QoS for Morocco
Telecommunications
Every operators are responsible for QoS
Fixed and mobile
Operators
Fixed Operator with
narrow mobility
Fixed and mobile: oligopoly
MAROC
TELECOM
MEDITELECOM
MAROC
CONNECT
(WANA)
VSAT
Operators
GMPCS
Operators
oligopoly
Oligopoly
SPACECOM
EUROPEAN
DATACOM
GLOBALSTAR
ORBCOMM
SOREMAR
THURAYA
Cotonou, Benin, 16-17 July 2012
Duopoly
INQUAM
MORATEL
GULFSAT
CIMECOM
Radio Trunk
Operators
ISP
Open
Competition
MAROC
TELECOM
MAROC
CONNECT
(WANA)
MEDITELECOM
ISPs
The QoS for telecommunications in
Morocco
A national
regulatory
Framework:
-Allocations
-QoS Obligations for
operators
Use feedback :
-Complaints,
-Press,
-End Users
Associations,
-…
Cotonou, Benin, 16-17 July 2012
International regulatory
framework (standards) :
-IUT-T : series E, G, P, Y,
QoS handbook, ...
-Regional : ETSI (series
EG), IEEE,…
The ANRT
regularly
monitors the
QoS
Ground
Measurements
(campaigns) by
ANRT
A benchmark of
best practices and
technological
monitoring
Analysis of data
received from the
operators in terms
of performances,
KPI and QoE
measurements
The QoS for telecommunications in
Morocco
Development of voice mobile telephony
service in Morocco
Cotonou, Benin, 16-17 July 2012
Presentation Scope
The QoS for Morocco telecommunications
A protocol for voice QoS measurements:
Choice of a subjective method,
List and definitions of indicators,
Measurements Methodology:
Definition of voice QoS measurement,
Sample and Statistic considerations,
Organization and Operation,
Process of Outcomes.
Recommendations to regulators/Op.
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements
Choice of a subjective method
The quality of service sets a premium on
models for reference (IUT-T standards : E,
I, G, P, Y… ou ETSI : ETSI TS 102 250-x),
on the levels of performance for service
provided and the measurement methods.
The models and methods are supposed to
be coherent with the users expectations.
For voice service, QoS requirements could
be related to many aspects sumed up in its
availability, accessibility and continuity and
reliability.
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements
Choice of a subjective method
Operator’s vision
Motivation
Center of
interest
(users)
Economic
Competitive
Regulator’s Vision
Rules and Regulations
Users loyalty (churn) and
attraction of newcomers
(Concept SLA)
Protection for users
Measurement
Local (various components of the
network)
KPI
External (end to end:
service as daily
understood by users)
Measurement
Tools
- Local machines meters in the
network (objectives methods)
- DTS
(objectives methods)
- Measures and
investigations by
subjective methods
- DTS
(Subjective Methods)
Necessity to harmonize the definitions of indicators between operators and the regulator.
Cotonou, Benin, 16-17 July 2012
Presentation Scope
The QoS for Morocco telecommunications
A protocol for voice QoS measurements:
Choice of a subjective method,
List and definitions of indicators,
Measurements Methodology:
Definition of voice QoS measurement,
Sample and Statistic considerations,
Organization and Operation,
Process of Outcomes.
Recommendations to regulators/Op.
Cotonou, Benin, 16-17 July 2012
Protocol for Voice QoS measurements
List and definitions of indicators
Rate
Indicator
Definition
TR
Rate of successful
communications and
maintained during 2 minutes
A communication is considered as successful if the call launched succeeds at the first attempt and
if the communication is maintained 2 minutes without being cut off. The rate (TR) is related to the
number of successful communications and the total number of call attempts made.
Perfect
Rate of successful
communications and
maintained during 2 minutes
and quality of perfect audio
A communication is considered as successful and the perfect quality if it is succeeded at the first
indicator and the audio quality considered by both parties are perfect (compared to the quality of
direct communication).
Acceptable
Rate of successful
communications and
maintained during 2 minutes
and quality of acceptable
audio
A communication is considered as successful and the acceptable quality if it is succeeded at the
first indicator and if audio quality considered by both parties is slightly disrupted without interferring
the conversation.
Poor
Rate of successful
communications and
maintained during 2 minutes
and quality of poor audio
A communication is considered as successful and poor quality if it is succeeded at the first indicator
and if the audio quality considered by both parties is frequently disrupted in the receiving channel
by numerous hit on the line, but it is possible again to hear one another.
Bad
Rate of successful
communications and
maintained during 2 minutes
and quality of bad audio
A communication is considered as successful and the quality degenerated if it is successful at the
first indicator and if it is very difficult to hear one another, the conversation is impossible.
RC
Rate of cut off
communications
A communication is considered as cut off if at the first attempt, it is established and maintained
more than 5 seconds, but cut off before 2 mins. The rate (TC) is the ratio between the number of cut
off communications and the total number of call attempts made.
RF
Rate of failed
communications
A communication is considered as failed if at the first attempt, it is not established or has not been
maintained, more than 5 seconds. The Rate (TE) is the ratio between the number of failed
communications and the total of call attempts made.
TR
TR + TE + TC = 100% and TNR = TE + TC
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements
List and definitions of indicators
Evaluation subjective de la qualité auditive : définitions
des défauts de communication vocale
Default
Definition
Low Voice
Level received more and less very low than the normal level
Echo
Return of its transmission distinctive from when it’s simultaneous
Metallic Noise
Chopping Noises to be compared to metallic one
Frying noise
Interferences
Micro-defaults
Small cut-offs of sound giving an impression of chopping
Silence
Blank enough long
Distorsion
Degradation of voice distant without other noise
Cotonou, Benin, 16-17 July 2012
Presentation Scope
The QoS for Morocco telecommunications
A protocol for voice QoS measurements:
Choice of a subjective method,
List and definitions of indicators,
Measurements Methodology:
Definition of voice QoS measurement,
Sample and Statistic considerations,
Organization and Operation,
Process of Outcomes.
Recommendations to regulators/Op.
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements
Measurements Methodology
A QoS measurement shall necessarily be done within the
coverage area by all concerned mobile networks. The QoS
measurement is different from the coverage
measurement.
A voice QoS measurement consists in trying to establish a
communication from the mobile investigator to the fixed
investigator to verify the reception of the call (no failure)
and the continuation of the communication for two
minutes (no cut off) and to assess the audio quality of
the established communication.
Each network is tested by a “couple” of investigators, on
mobile and the other fixed.
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements
Measurements Methodology
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements
Measurements Methodology
A communication is considered failed if:
 at the first attempt (maximum delay time is about
15 seconds), it is not established,
 or has not been maintained more than 5 seconds.
A communication is considered cut off if, at the first
attempt:
 it is established and maintained more than 5
seconds,
 but cut off before 2 minutes.
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements
Measurements Methodology
Precautions to be taken:
Outdoor : avoid measuring under trees.
Indoor : very inside buildings and till 8th floor.
Incar : use terminals without additional antenna and
drive at a speed less than 80km/h.
Force terminals on 2G in case of use of bi-mode posts.
Every points for measurements shall be covered.
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements
Measurements Methodology
Sample and statistic considerations:
Average duration of a measurement: 7 mn,
Average duration for measurements takes
place everyday: 8:00 to 10:00 (Example :
09:00-13:00 and 14:00 to 20:00),
Everyday (per team) : 85 measurements x
Number of operators; For 3 operators : 255
measurements per day,
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements
Measurements Methodology
Sample and statistic considerations:
Breakdown Element: Per operator, Per site
(cities, motorways, national highways,
railways), per configuration (Indoor, Outdoor,
Incar), per traffic destination (OnNet and
OffNet).
Important numbers to provide a significant
sample (statistic error about 2%) :
elementary values after any breakdown shall
be higher than 33.
Cotonou, Benin, 16-17 July 2012
Presentation Scope
The QoS for Morocco telecommunications
A protocol for voice QoS measurements:
Choice of a subjective method,
List and definitions of indicators,
Measurements Methodology:
Definition of voice QoS measurement,
Sample and Statistic considerations,
Organization and Operation,
Process of Outcomes.
Recommendations to regulators/Op.
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements
Measurements Methodology
Organization and Operation:
Decider to achieve based on own resources
or controlled subcontracting,
Select effectively investigators (mobiles and
Fixed) and conduct a targeted training,
Consult with operators on the methodology
but not the sample and the planning of each
campaign,
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements
Measurements Methodology
Organization and operation:
Develop a guide for the investigator with
some templates to be used by the
investigator: Ref., delay, GPS Information,
recipient, outcomes,…
Use the principle for codificationprincipe de
codification : opérateur, site, configuration,
trafic,…
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements
Measurements Methodology
Organization and operation:
Guarantee any conditions so that each
measurement is exactly conducted in the
same conditions for every operators: delay,
space, method,…
Proceed to a number of important blank tests
calibrate any test system.
Cotonou, Benin, 16-17 July 2012
Presentation Scope
The QoS for Morocco telecommunications
A protocol for voice QoS measurements:
Choice of a subjective method,
List and definitions of indicators,
Measurements Methodology:
Definition of voice QoS measurement,
Sample and Statistic considerations,
Organization and Operation,
Process of Outcomes.
Recommendations to regulators/Op.
Cotonou, Benin, 16-17 July 2012
Protocol for voice QoS measurements
Measurements Methodology
Outcomes Processing: (Scheduled to be
Delivered)
Trend in indicator “positive” : successful rate.
Trend in indicator “negative” : unsuccessful
rate = TE+TC.
Reports:
Global Report: reference,
Specific Report to each operator (request for
actions plan for corrections),
Report for publication.
Cotonou, Benin, 16-17 July 2012
Presentation Scope
The QoS for Morocco telecommunications
A protocol for voice QoS measurements:
Choice of a subjective method,
List and definitions of indicators,
Measurements Methodology:
Definition of voice QoS measurement,
Sample and Statistic considerations,
Organization and Operation,
Process of Outcomes.
Recommendations to regulators/Op.
Cotonou, Benin, 16-17 July 2012
Conclusions and Recommendations
Recommendation n°1
Permanent Consultation with the upstream
operators upon the methodology: financing
of campaigns. Adoption of an agreed protocol
of references for measurements.
Recommendation n°2
Positive use of outcomes by the operators
(consider sanctions as last resort).
Cotonou, Benin, 16-17 July 2012
Conclusions and Recommendations
Recommendation n°3
Publication of measurements outcomes,
Proposal of a scenario :
1st voice campaign
on based upon a
Communication of operators’
large sample (thirty
results
cities, highways,
national motorways
and railways).
Delay of 5 to 6 months
1st quarter of the
for correction of discrepancies
year
identified by the ANRT
Cotonou, Benin, 16-17 July 2012
2nd similar campaign
practically based on
the first upon the
same sample.
4th quarter of the
year
(Publication of results)
Conclusions and Recommendations
Recommendation n°3 (bis)
Publication of outcome measurements
TMR×0,95
TMR×0,98
TMR×0,99
TMR×1,01
TMR×1,02
TMR×1,05
TMR
---
--
-
=
=
+
++
+++
Taux de réussite global (On-Net et Off-Net) par site de mesures et par opérateur
Site for
measurement
s
All cities
TMR
Operator 1
Operator 2
Operator 3)
95,32%
=
=
=
Cities
96,78%
=
=
=
Highways
95,68%
=
--
+
National
Motorways
96,03%
+
=
=
Railways
87,00%
=
=
=
The TMR is equal to the successfuk average rates (TR) recorded by each of the three operators:
Cotonou, Benin, 16-17 July 2012
TMR
Thanks for your attention
Questions/Answers
talib@anrt.ma // htalib@ties.itu.int
Cotonou, Benin, 16-17 July 2012