International Telecommunication Union
ITU-T
Quality Assurance for IPTV
David Hands
BT
Research and Venturing
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
Overview
ITU-T
o Introduction
o Perceptual Quality Measurement
o NR perceptual quality method
o Quality Assurance for IPTV
o BT QA Tool
o Demonstration
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
2
Quality Assurance and Perceptual Quality
ITU-T
o
Standard QA tools exist for TV services
• Perform syntax checking + Post-decode
analysis (e.g. bit-rate, frame rate)
• Can include basic quality index
o
Such tools do not include perceptual
quality measurements
o
BT has developed a QA tool for IPTV
Tool is based on accurate, real-time
perceptual quality measurements
o
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
3
Objective Quality Measurement
ITU-T
o Quality measurement:
• Objective
— Pure computational
— Network performance
• Objective perceptual
— measurements representative of human
perception
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
4
Objective Perceptual Quality Measurement
ITU-T
o Full Reference
o Reduced Reference
o No Reference
o Traditionally methods operate at pixel level
o BT has developed a novel bit-stream based
approach to objective perceptual quality
measurement
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
5
Bit-stream Based PQM
ITU-T
o Why bit-stream?
• Includes quality critical information
— quantiser step-size, DC/AC co-efficients, MB
skipping, motion vector values
• This is lost during decoding
• Computationally light … fast measurement
possible
• Specific to coding scheme – accuracy
advantages
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
6
BT NR Model
ITU-T
o
o
o
o
Measurement tool embedded in a H.264
decoder
Parser extracts parameter values from bitstream prior to decoding
Additional parameter set can be extracted
from decoded picture
This hybrid approach provides
1. information about encoding applied to
picture
2. picture properties following decoding
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
7
Performance
ITU-T
5
MOSp
4
3
Train
Test
2
1
1
2
3
MOS
4
5
r = 0.94
MSE = 0.4
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
8
Quality Assurance for IPTV
ITU-T
Video
encoding
Bit streams
pool
Ingestion
Decision
IPTV servers
ALERTS
Reports
Real time
streams
monitoring
Customers
Manual / Automatic
streams correction
Quality Assure
Content Preparation
Content Delivery
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
9
BT Quality Assurance Tool
ITU-T
o Provides perceptual video quality
o
o
o
o
measurement + audio analysis
QA tool includes user interface
Allows operator to select quality threshold
(according to ITU quality scale)
Additional thresholding can be invoked to
determine pass / borderline / fail content
Operator can examine quality of suspect
quality content
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
10
Demonstration
ITU-T
ITU-T Workshop on “End-to-End QoE/QoS“
Geneva, 14-16 June 2006
11