International Telecommunication Union
ITU-T
End to End QoS and Triple Play
Quality of Experience for Residential Voice,
Data and Video Services
Bruce Adams
Manager, Standards and Research
Telchemy, Incorporated
May, 2006
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
1
What is Triple play to the Consumer ?
ITU-T
o Service providers are increasingly offering multiple
services
• Cable Providers
• Data (e.g. US DOCSIS)
• Telephone Service (e.g. US PacketCable)
• Video
• Telephone Service Providers
• Data (DSL, Fiber to the home/ curb)
• Telephone Service (VoIP)
• Video (IPTV)
o Consumer simply wants service quality and reliability
to be as good as (or better than) it was before
o It SHOULD be as good
May, 2006
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
2
Should be ?!? Why not ?
ITU-T
o TDM Telephone is a dedicated – and hardened –
service – High QoS
o Broadcast and Cable television are dedicated
services – Hi QoS
o These are designed and deployed to be
single-purpose, high quality systems
BUT…
o VoIP and IpTV rely on General Purpose – and often shared infrastructures.
o Infrastructure failings: Shared or Insufficient bandwidth, link
failures and so on.
o IP’s inherent failings: Lost packets, network congestion, jitter
and so on.
o Analog portions – no control for providers
May, 2006
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
3
What do do?
ITU-T
o Service providers must monitor their entire network
o Consumer device manufacturers should embed and
enable system monitoring functionality in end devices
o Access and Gateway devices must provide
monitoring – and reporting- capabilities.
May, 2006
4
Triple Play Architecture - VoIP
ITU-T
Customer Premise
Trunking
Gateway
DSL
IP
Analog
phone
PSTN
May, 2006
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
5
Triple Play Architecture - IPTV
ITU-T
Customer Premise
Video
Content
IP Set Top
DSL
IP
May, 2006
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
6
Triple Play - Issues and Problems
ITU-T
Customer Premise
Video
Content
IP Set Top
DSL
Trunking
Gateway
IP
Link failures,
Route flapping
PSTN
Access link
congestion
Analog
phone
Echo
May, 2006
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
7
Typical IpTV implementation
ITU-T
Customer Premise
Video stream - H.264/MPEG 4 or VC1
At 500kbits/s - 6Mbits/s+
IPTV
Set top
VOD Server
Residential
Gateway
IP
Broadband
In-home
wiring
Real
Time
Encoder
Service provider responsible
for IP service quality
?
UDP or TCP transport path
May, 2006
8
IPTV Potential Problem Areas
ITU-T
Customer Premise
Home LAN
speed/ link
quality
VOD Server
VoD Server
Congestion
Access Link
Congestion
Congestion or
shaping within
IP network. Use
of RED by routers.
May, 2006
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
9
What is the range of problems ?
ITU-T
o Lost packets, and Jitter: Discarded packets
• Dropouts and PLC replacements in Voice
• Motion freezes and block artifacts in Video
o Long delays: Conversational difficulties
• IpTV is much more immune: One-way
o Varying delay: Jitter and Route Flapping
• Congestion due to access link and LAN
near occupancy limits.
• Automatic rerouting for load balancing
o Content Dependency
• Last game of the World Cup Finals
• Call home by traveling family member
o One-way: Passive versus Two-way: Interactive
• Video has leeway to increase protections that
telephony does not have.
May, 2006
10
How do the problems effect users ?
ITU-T
o VoIP:
• Jitter and lost packets lead to signal dropouts and PLC
replacements – More lost packets, lower quality.
• Delay leads to difficulty in normal conversational flow.
• Long delays can enhance Echo problems – an analog problem.
• Route flapping alters end-to-end delay, leads to gaps in speech,
Echo cancellation problems.
• Bandwidth constraints lead to use of lower bit-rate and voice-quality
codecs.
o IpTV
• Jitter leading to packet discard and Lost packets lead to noticeable
degradations in the picture – Visible blocks that persist until new IFrames are received.
• Lost I-Frames can cause halting of play out.
• Bandwidth constraints lead to use of lower bit rate and video-quality
codecs.
May, 2006
11
IP problems are transient
Ban d w id th (kb it/ s )
ITU-T
500
Congestion
400
Voice
Data
300
200
100
0
0
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18
0
1
2
3
4
5
6
7
8
9
10
5
MOS
4
3
2
1
11
12
13
14
15
16
17
18
Time
May, 2006
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
12
Perceptual effects of changing quality
ITU-T
Good quality most
of the time
5-8
seconds
15-30
seconds
Reported
Call quality
Poor quality during burst of
loss/discards
ETSI TS 101 329-5 Annex E
May, 2006
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
13
Multimedia Performance Monitoring Model
ITU-T
Arriving
packets
4 State Markov Model
Gather detailed
packet loss/ discard
distribution info
in real time
Loss/ Discard
events
Jitter
Buffer
(or emulator)
CODEC
May, 2006
CODEC,
Payload
parameters
Metrics
Calculation /
Perceptual
Model
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
Perceptual
Quality scores
and
Diagnostic data
14
Application to VoIP - e.g. VQmon
ITU-T
o Support Narrowband and Wideband codecs
o Report Listening and Conversational Quality
o Superset of ITU G.107 and ITU P.VTQ (P.564)
• Incorporates time varying impairment model
• Incorporates signal level, noise level, echo level
• Non-linear impairment combination model
o Low processing power, small size…
• Complexity 500 “IPS”
May, 2006
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
15
VQmon 2.0/2.1 computational model
ITU-T
Perceptual
model
Burst loss
rate
Calculate
R-LQ
MOS-LQ
Ie mapping
Gap loss
rate
Recency
model
Signal level
Noise level
Remote ACOM
SOWD
May, 2006
Alerting
Calculate
Ro, Is
Calculate
Id
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
Calculate
R-CQ
MOS-CQ
16
Application to IPTV
ITU-T
o Supports MPEG1/ 2/ 4, H.261/3/4 codecs
o Supports various combinations of transport protocol
• RTP, MPEG-TS, MPEG-TS over RTP….
o Low complexity
• 0.01-0.1 MIPS/ stream, depending on packet rate
May, 2006
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
17
VQmon/SA-VM - computational model
ITU-T
Burst loss
rate
Perceptual model
Ie mapping
Gap loss
rate
Calculate
Transmission
Quality
Factor
Video Codec
Parameters
Frame Rate
Bit Rate
Calculate
Vo, Vs
Video Content
Factor
Calculate
PQSbase
May, 2006
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
Calculate
Picture
Quality
Score
18
Measurement performance - VoIP
ITU-T
Co m p a riso n o f V Q m o n v s ACR M O S - G. 7 1 1 n o PLC
Co m p a riso n o f V Q m o n v s ACR M O S - G. 7 1 1 w it h PLC
5
5
ACR MOS
ACR MOS
4.5
4.5
VQm on MOS- LQ
4
3.5
3.5
M O S Sco re
M O S Sco re
VQm on MOS- LQ
4
3
2.5
3
2.5
2
2
1.5
1.5
1
1
0
5
10
15
20
0
5
Pa ck e t Lo s s Ra t e ( % )
10
Co m p a riso n o f V Q m o n v s ACR M O S - I LBC 1 5 . 2 k
20
Co m p a riso n o f V Q m o n v s ACR M O S - G. 7 2 9 A
5
5
ACR MOS
ACR MOS
4.5
4.5
VQm on MOS- LQ
VQm on MOS- LQ
4
4
3.5
3.5
M O S Sco re
M O S Sco re
15
Pa ck e t Lo s s Ra t e ( % )
3
2.5
3
2.5
2
2
1.5
1.5
1
1
0
5
10
15
Pa ck e t Lo s s Ra t e ( % )
May, 2006
20
0
5
10
15
20
Pa ck e t Lo s s Ra t e ( % )
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
19
Measurement Performance - IPTV
ITU-T
NTIA VQM full reference Video MOS
5.0
4.5
4.0
3.5
3.0
2.5
2.0
1.5
1.0
1
1.5
2
2.5
3
3.5
4
4.5
5
VQmon estimated Video MOS
May, 2006
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
20
Impact of time varying problems
ITU-T
Est i m at ed Sub j ect i ve T est Scor e
4
l
ea
d
I
VQmon - models impact
of transient IP problems
3.5
3
2.5
2
“E Model” - based
on average packet loss
rate
1.5
1.5
2
2.5
3
3.5
4
Actual Sub j ective Test Scor e
Independent test by France Telecom and University of Bochum
May, 2006
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
21
Where to measure?
ITU-T
o Premises: Consumer equipment
•
•
•
•
•
•
IP Phones and residential gateways
Set top Boxes and Digital Tuners.
Home routers
Cable Modems and DSL modems
Soft phones (PC Based)
Cellular phones
o Network: Carrier Equipment
• DSLAM, Gateways, Amplifiers, Head ends, CMTS,
Switches, Session Border Controllers, IADs, Routers, ……
o Probes
• In-network stand-alone probes
• Field service equipment
May, 2006
22
Triple Play - Monitoring Architecture
ITU-T
Customer Premise
Video
Content
Trunking
Gateway
PSTN
IP Set Top
DSL
IP
Analog
phone
Real time reporting
Through RTCP XR and SIP
VoIP Monitoring Function
IPTV Monitoring Function
May, 2006
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
23
Final thoughts
ITU-T
o Carriers delivering services over their own
infrastructure
• Better managed, fewer issues
• Can identify problem areas
o Content and service providers using infrastructure
provided by others – Internet
• Not managed: more problems,
• Still, can locate problem areas
o The future: Services to the User not the Location
• Problems will vary depending on user location
• Same tools can be used irrespective of location
May, 2006
24
Summary
ITU-T
o VoIP and IPTV Deployments can suffer from
performance problems
• Transient IP problems due to congestion
• Content/ environment/ encoding problems
o Monitoring functionality can be embedded directly
into residential gateways, IP set tops…..
• Least cost, closest to the customer
o Protocols and performance measurement technology
already exists
• Mix of ITU standard, extended standard, proprietary
algorithms
• Already available in silicon
May, 2006
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
25
International Telecommunication Union
ITU-T
Thanks!
Questions?
May, 2006
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
26