Monitoring and Benchmarking of QoS/QoE of
Multimedia services in mobile networks
(Buenos Aires, Argentina 24-25 July 2014)
Monitoring QoS/QoE of mobile
networks –using NMS
(the most effective approach)
Tahitii Obioha
Network Planning and Optimisation Engineer
Planet Network International, France.
Buenos Aires, Argentina, 25 July 2014
Quest for a Monitoring Tool
Monitor simultaneously 24H/7 the state of all network
elements/acess technologies in an operator network passively
without compromising their daily operations
Put a check on service degradation and sometimes outages to the
benefit of the government and the masses.
Quest for a system that can monitor the performance of mobile
operators independent of the vendor and technology
Flexible Benchmark Audit; weekly, monthly or quarterly.
Track the evolution and expansion of the mobile operators network.
Reduce churn rate of end-users directly or indirectly
A robust system that is easily upgraded to meet the rapid evolution
of mobile technology.
A platform that will facilitate actionable decision making to the
benefits of all parties involved with little or no OPEX.
Create a level playing field for all operators and protect the interest
of all consumers not just those in the urban areas.
Solution that would relieve Regulators of their daytime headaches
and nighttime worries.
Buenos Aires, Argentina, 25 July 2014
2
Why Monitor QoS/ QoE of mobile networks
Monitoring = observing, checking and keeping a continuous
record of the progress or quality of something.
It is the use of any available technical tool to assess permanently
or for a given period of time a particular QoS parameter, e.g. call
set up time. (ETSI EG 202 009-2)
Reasons
For end-users : Satisfaction, get the best value for their money
amidst others
For operators : end-user Satisfaction, Increase revenue,
achieve customer loyalty, maintain competitive
edge amidst others. Surveys have shown that
about 90% of customers will not complain before defecting.
For Regulators: protect consumers, ensure end-user satisfaction,
create a level playing field for operators to compete amidst others.
They all share a common Objective: CUSTOMER SATISFACTION
Buenos Aires, Argentina, 25 July 2014
3
Relationship between NP,QoS and QoE
NP
QoS & QoS
QoE
NP is a subset of QoS which in turn, is a subset of the overall QoE scope
NP parameters ultimately determine the QoS
QoE can
vice
be
predicted
versa
Buenos Aires, Argentina, 25 July 2014
from
a
(ETSI
given
set
of
TS
QoS
param
102
4
Relationship between QoS and QoE
There is an exponential relationship between QoS and QoE
End-to-end QoS is an important enabler for QoE in otherwords
a better network QoS in many cases will result in better QoE
QoE
vice
can
be predicted
versa
from a
(ETSI
given
set
TS
of
QoS param
102
QoE may be improved with the right QoS mechanisms.
Buenos Aires, Argentina, 25 July 2014
5
QoS Measurement
NP
QoS
Objective
Active
Non-NP
Subjective
Passive
Monitoring
Intrusive
NonIntrusive
Surveys, churnrate, etc…
Walk/Drive
Test Around
test
OMC-R
Counters
using NMS
Ex.NMS: RPM System, Net Act,
imanager M2000 …
Testing
Ex. DT Tools: Nemo, XCAL, TEMS …
QoS/QoE Assesement
TWO DIFFERENT AND COMPLEMENTARY APPROACHES TO MOBILE
QoS (ETSI EG 202 057-3)
Walk/Drive- round tests
Measurements based on network element counters using NMS
QoS Assesment Target
Best Suitable QoS
Approach(es)
One-time snap shot
DT
Acceptance Procedure
DT/ NMS
Continuous Monitoring
NMS
Optimisation Cycle
DT + NMS
Buenos Aires, Argentina, 25 July 2014
7
Difference Between NMS and DT
Network Management System
Drive Test (DT)
QoS experienced by users are taken into account as
the calls are made with the real terminals by the real
users.
The QoS experienced is not taken into account by
users depends to some extent on the design of their
terminals which may differ somewhat from those
used for formal tests.
Representativity is assured as real traffic provides
measurements from the entire network 24h/7
Drive test does not provide a representative of the
performance of the whole network because of
variation in user density and the complexity of
providing service in different areas. (60% of calls
made in a network are made indoor).
Track performance from cell level to Core NE
Performance stops at Base station level
Provides comparabilty of Congestion and network
failures
Misses out on planned Site outages and network
failures.
Follow up on Operators’ network expansion, type of
equipment deployed and mobile evolution
Impossible to track Network Expansion using DT
NMS reports are unbiased, reliable and actionable as
it tracks the availability of network elements (Ex.
BSC/RNC)
DT reports are simply informative and sometimes
biased.
Indepedent of enviromental factors
Depedent on environmental factors such as weather
conditions, seasons, location, timing etc
Traffic carried over the network and associated grade
of service (GoS) are obtained using this approach
DT has no information on Traffic carried over a given
network.
KPI Build flexibility from numerous counters of NEs
Limited KPI observation as it measures the network
from an external point.
Relatively no OPEX involved
High OPEX as well as time and Labor intensive
Passive and non- intrusive, best approach in
continious QoS monitoring.
Active and intrusive, best approach for QoS
(coverage) benchmarking purposes.
8
Draw backs of DT illustrated
DT is impraticable, owing to narrow
streets, houses built on top of others.
Buenos Aires, Argentina, 25 July 2014
9
Draw backs of DT/WT illustrated
WT is also impraticable, as it is labor intensive giving the
size of the area that needs to be covered
Buenos Aires, Argentina, 25 July 2014
10
Draw backs of DT illustrated
Buenos Aires, Argentina, 25 July 2014
11
Draw backs of DT illustrated
Buenos Aires, Argentina, 25 July 2014
12
TMN Architechture
The Five-Layer TMN Network Management Architecture
(ITU-T Rec M. 3010)
Buenos Aires, Argentina, 25 July 2014
13
NMS OVERVIEW
All events on the network are time stamped and logged
24/7/365
Buenos Aires, Argentina, 25 July 2014
14
NMS CHALLENGES
There are lots of proprietary protocols besides CMISE at
inception but CORBA and SNMP are some of the OPEN
Standard protocols employed in NML-EML interface
Buenos Aires, Argentina, 25 July 2014
15
NMS OVERVIEW
60% of calls made in a given network are made Indoors
(offices, homes, malls, shopping centres, etc)
Buenos Aires, Argentina, 25 July 2014
16
RPM SYSTEM or
NMS OVERVIEW
any other third party
NMS
Monitoring tool
All events(FCAPS Mgt files) on the network are logged and
backed up at the OMC storage servers.
Buenos Aires, Argentina, 25 July 2014
17
NMS OVERVIEW
Needless connecting to all the interfaces in an operator’s
network using relatively expensive probes.
Buenos Aires, Argentina, 25 July 2014
18
NMS OVERVIEW
Regulators’
NMS/Monitoring
platform
Secure FTP
All events(FCAPS Mgt files) on the network are logged and
backed up at the OMC storage servers.
Buenos Aires, Argentina, 25 July 2014
19
PM FILE COLLECTION
Regulators’ are interested only in PM files thus an FTP
server can be securely configured to automatically retrieve
and process PM files at the OMC storage servers.
Buenos Aires, Argentina, 25 July 2014
20
Network Management Systems
Measurements performed on real traffic conditions and therefore as expected
gives a more realistic vision of the QoS. There are CM FM PM files logged at
the OMC 24/7/365.
EXAMPLES OF NMS solutions in the market today
For REGULATORS (Vendor-Independent)(Focus only on PM files with
mechanism for levying penalties)
Regulators (QoS)Performance Management (RPM)system- compatible
with all major network vendors (Huawei, ZTE, Ericcsson, Alcatel and NSN)
FOR OPERATORS (Vendor –dependent)(focus on CM FM and PM files)
imanager M2000 for Huawei vendor
NetNumen U31 for ZTE vendor
NEtAct for Nokia Siemens Vendor
NetOp EMS for Ericsson and Omnivista 2500 for Alcatel vendor
NOVA (vendor independent focus on FCAPS)
Buenos Aires, Argentina, 25 July 2014
21
RPM SYSTEM OVERVIEW
Regulators’ (QoS) Performance Management System is the novel NMS solution that
handles the task of interfacing all the operators’/service providers’ network
monitoring systems, collect perfomance data records and create KPI reports that
renders a given network performance against published benchmarks.
Main Features
• QoS monitoring and reporting 24/7 all year round
• Unlimited number of Users/ Single License
• Multi vendor Multi access technology
• Automatic Reporting and 3D GIS
display(compatible with google earth
• Friendly and intuitive web user interface
• No running costs and upgrades easily as mobile
access technology evolves
• Built in compliance with ITU-T Series O and M
ETSI EG 202 057-3 and ETSI EG 202 009-2&3
Buenos Aires, Argentina, 25 July 2014
22
EFFECTIVE QoS MONITORING IN PLACE
Deploy Rapidly..Customise easily
Buenos Aires, Argentina, 25 July 2014
RPM system is conformant to all the
following International ISO/TMN
standards
X.700, X.711, X.721, X.733
Q.811, Q.812, Q.821, Q.822
M.3000, M.3010, M.3100, M.3200,
M.3400
G.774 series
23
Conclusion
The assessment of the QoS/QoE is expected to be evaluated in
checking criteria against reference values. These criteria are
measured either objectively via technical means or subjectively via
surveys amongst the users. Experts agree that a mix of
objective(intrusive and non-intrusive) and subjective
measurements remains the best means to get the whole QoS
picture.
Today, four out of five Regulators only employ Drive Test(intrusive)
both for one-time snap shot, benchmarking and continuous QoS
monitoring(reactive sense). Given DTs drawbacks and inaccuracy
with respect to continuous QoS monitoring, its high-time
Regulators, gaining from operators’ perspective, adopted the novel,
practical and more efficient approach – the use of Network
Management Systems (NMS) for proactive continuous QoS
monitoring.
Buenos Aires, Argentina, 25 July 2014
24
Recommendations
Regulators, as recommended in ETSI EG 202 009-1
clause 7, in the quest for effective continuous QoS
monitoring and assessment should perform the
measurements themselves and make the results
available publicly rather than entrust in mobile
operators to provide the QoS information
Thus Regulators should procure RPM system or any
other NMS that facilitates the complex task of
enforcing mobile operators to adhere to agreed
KPIs, put a check on service degradations /outright
outages and even levy penalities(as advised in ETSI
EG 202 009-3) for repeated failure to achieve
minimum QoS/ QoE targets.
Buenos Aires, Argentina, 25 July 2014
25
THANK YOU FOR YOUR
ATTENTION
For more information on RPM
SYSTEM or any other NMS
Feel free to contact me
or email
cblanchard@planetworkint.com
Buenos Aires, Argentina, 25 July 2014
26