ITU Workshop on “Monitoring and Benchmarking
of QoS and QoE of Multimedia Services in
Mobile Networks”
(Dubai, UAE 2-3 November 2014)
Linking QoE and QoS on data services
Esa Vesterinen
Vice President
Omnitele
esa.vesterinen@omnitele.com
Content
• QoS – QoE framework for data
services
• Case studies:
•
•
•
WWW browsing
Network performance vs YouTube experience
Optimisation
©Omnitele Ltd. 2014
2
our company
PIONEERING
INDEPENDENT
CONSULTING
founded in 1988 to design and rollout world’s first GSM
network, 1000+ projects in 80+ countries since then
owned by Finnish telecom investors, independent
of all operator groups and equipment vendors
consulting and professional services for telecom
service providers and regulators
consulting for mobile industry since 1988
our services
Technology
Strategy
Design and
Optimisation
Audit and
Benchmark
Performance
Management
maximised customer experience
minimised network cost
©Omnitele Ltd. 2014
4
what are QoS and QoE?
price
QOE DEPENDS ON…
 expectations
 branding
 socio-economic
background
 price
 customer care
 provisioning
 end-user QoS
network
QoS
5
brand
service
device
©Omnitele Ltd. 2014
QoE
session focus
expectations
QoE, QoS and network performance quantified
QUALITY OF EXPERIENCE
how well service quality meets expectations?
QUALITY OF SERVICE
measured application/service KPIs
NETWORK PERFORMANCE
capacity, coverage, functionality, efficiency…
©Omnitele Ltd. 2014
6
call completion rate sms send time sms
completion rate file transfer time www page waiting time www
page success rate video buffering time
video setup success rate
call setup time
RSCP
Ec/N0
latency
jitter
packet loss
RSRP
RLC throughput
RSRQ
RAB setup success rate
modulation
coding
TX power
G-factor
Channel C/I
Timing Advance
Mobile TX Power
Voice Codec Usage
Handover Success Rate
MCS Usage Distribution
Time Slot Utilization
Block Error Ratio
Active Set
Size
SHO Success Rate
ISHO Success Rate
CQI
E-DPDCH Throughput
PDSCH
modulation
MAC DL BLER
MAC UL BLER
MAC UL Retransmission Rate
Common challenge: “How is the quality?”
???
“network is excellent…”
“…customers not satisfied”
need to find right targets for acceptable QoE!
©Omnitele Ltd. 2014
7
OPERATOR CASE: CEM FOR WWW BROWSING
Let’s take a dive into WWW
browsing Quality of Experience…
©Omnitele Ltd. 2014
8
sufficient web browsing
experience for high value customers
OBJECTIVE |
1. QoE targets: define the desired WWW QoE level
2. QoS targets: measure QoE & QoS and crosscorrelate to define the relation
3. CTO Targets: Link QoS with NW performance to
find right network performance KPI targets
©Omnitele Ltd. 2014
9
1. QOE TARGET |


define the desired WWW QoE
QoE defined in MOS scale
Captured with end-user queries:
“On 1-5 scale, what is your
WWW browsing experience?”
MOS
QoE
Impairment
5
Excellent
Imperceptible
4
Good
3
Fair
Slightly Annoying
2
Poor
Annoying
1
Bad
Very Annoying
©Omnitele Ltd. 2014
10
Perceptible but not annoying
SUFFICIENT
QoE TARGET
90% of high value
customers MOS ≥ 4
2. QOS TARGET |
correlate QoS with QoE
Web Browsing QoS and QoE
2.18
5
MOS
QoE [MOS]
3.74
Average(s)
4
6.02
unsatisfactory region
3
9.99
(MOS < 4)
2
13.83
1
0
2
4
6
8
10
12
14
16
QoS: WWW page waiting time [s]
QoS target: WWW download time < 3s
©Omnitele Ltd. 2014
11
18
20
3. CTO TARGETS |
find required NW performance
QoS: WWW page waiting time [s]
1000KB WWW Page Waiting Time vs. Bitrate
HSPA
10
LTE
unsatisfactory region
(www download time > 3s)
8
6
accepted region
4
(www download time < 3s)
2
0
0
1
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20
NW Performance: Bitrate [Mbit/s]
©Omnitele Ltd. 2014
12
CTO
TARGET
90% of subs get
>3.5 Mbit/s in 4G
>4.5 Mbit/s in 3G
Case summary
1. QoE targets: define the desired WWW QoE
level
call completion rate sms send time sms
2. QoS
measure QoE & QoS and crosse transfer time www page waiting
time targets:
www
correlate
to define the relation
video buffering time video setup
success rate
jitter
packet loss
RSRP
RLC throughput
RSRQ
modulation
coding
TX power
G-factor
Channel C/I
TX Power
Voice Codec Usage
Handover Success Rate
Time Slot Utilization
Block Error Ratio
Active Set
ISHO Success Rate
CQI
E-DPDCH Throughput
PDSCH
L BLER
MAC UL BLER
MAC UL Retransmission Rate
3. CTO Targets: Link QoS with NW performance
to find right network performance KPI targets
©Omnitele Ltd. 2014
13
MOS ≥ 4
WWW DL time
<3s
NW performance
>3.5 Mbit/s in 4G
>4.5 Mbit/s in 3G
OPERATOR CASE: CUSTOMER EXPERIENCE BENCHMARK
competitive positioning of
YouTube experience
©Omnitele Ltd. 2014
14
4G smartphone benchmark in Netherlands
big difference in NW performance…
…thin margins in customer experience
bitrate [Mbit/s]
buffering time [s]
40
2
30
1.5
20
1
10
0.5
0
0
T-Mobile
Vodafone
KPN
T-Mobile
Vodafone higher bitrate,
but KPN faster YouTube
©Omnitele Ltd. 2014
15
Vodafone
KPN
OPERATOR CASE: CUSTOMER EXPERIENCE OPTIMISATION
improving QoE for Kuwaiti operator
©Omnitele Ltd. 2014
16
objective: customer experience improvement
1.
Measure end-user QoS of
smartphone voice and data
2.
Measure network performance
3.
End-to-end troubleshooting &
improvement recommendations
©Omnitele Ltd. 2014
17
customer centric optimisation area prioritisation
number porters
tickets
landmarks
population
©Omnitele Ltd. 2014
18
coverage plots
churn
planned sites
complaints
improved customer experience with zero CapEx
15%-25% faster browsing in optimised areas
10% of sites optimised, zero capital expenditures
end-user QoS and network performance connected
©Omnitele Ltd. 2014
19
recap & takeaway messages
1
common challenge: CTO team & CMO
team targets often not fully aligned
2
linking network performance to customer
experience challenging but not impossible
3
©Omnitele Ltd. 2014
20
proper CEM framework results in maximised
customer experience & minimised network cost
www.omnitele.com
©Omnitele Ltd. 2014
21