Streaming Performance over Third Generation
Cellular Networks
Author: Musa Aldarawsheh
Senior I&V Engineer/ Nokia Networks
Supervisor: Prof. Sven-Gustav Häggman
Instructor: M.Sc. Karl Tigerstedt
1
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Contents
Introduction
PS streaming QoS theory
QoS Key Performance Indicators
Environment Description
Measurements Tools
Preliminary Streaming QoS Tests
Streaming Measurements Using Nokia Internal Streaming Server
Streaming Measurements Using Helix Universal Server-Mobile
Streaming QoS Measurements over Real Time Bearers in Unloaded Cell
Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell
Streaming in Highly Loaded Cell With Conversational Class Users
Streaming Performance Measurement in Mobile Situation
GPRS Attach and PDP Context Activation Times
Summary & Conclusions
References
2
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Introduction
•
This work is aimed to verify the performance of multimedia streaming service over a WCDMA network
from end-user perspectives. The measurements were carried out over streaming and interactive QoS class
in stationary and mobile situation, the mobility measurements include streaming during soft handover.
•
The streaming QoS will be verified through a set of Key Performance Indicators (KPI). These includes call
setup and call quality KPIs.
•
Multimedia streaming verification was carried out in Nokia WCDMA Test Network, in Espoo (Finland) during
weeks 45 - 52 2004.
3
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
PS Streaming QoS Theory
•
Multimedia streaming is a technology that allows compressed live or pre-recorded video and audio content
to be delivered to the end user [Lai_p398]. Streaming service considered as real time service because the
user plays back the content while downloading. Examples of streaming services are video-on-demand ,webradio, and news streams.
•
Media can be divided into two types continuous and discrete media,The first type is only handled in this
report
To establish a streaming session there are different protocols involved to set-up the connection between the
streaming server and the streaming client, They are :
•
•
RTSP (Real Time Streaming Protocol) is used to for session set-up and session control
RTP (Real Time Protocol) is used to transport media data (video/audio) over unreliable connection
•
(UDP/IP).
RTCP (Real Time Control Protocol) used to carry information on the participants and session quality.
RLC transfer modes for PS Streaming
•
RLC operates in one of three modes. These are Acknowledge mode (AM), Unacknowledged mode (UM) and
transparent mode(TM).
•
The possible RLC transfer modes for PS Streaming services are either UM-Unacknowledged mode or AMAcknowledged mode
Token Bucket Algorithm:
Token bucket algorithm is a reference algorithm to conform the definition of bit rate, see [23107].
4
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
QoS Key Performance Indicators
•
QoS is the ability of the network to provide a service with assured service level to the user of the network
•
QoE (Quality of user Experience) is how an end user perceives the usability of a service when on use
(how satisfied he/she is with the service usage)
•
The following QoS KPIs has been defined to measure the performance of multimedia streaming over
WCDMA networks.
•
Session setup time :This is the time period when the streaming client requests media content from the
streaming server until the time when the session setup between client and server is ready and the
media start to flow.
•
Initial Buffering time :This is the time elapsed between the completion of session set-up and the time
that media starts playing at the client's device. This time is dependent on client memory storage.
•
Total set-up time :
This is the time when the user request a media content form the streaming server until the time the
media is starting to be displayed in the streaming client.
•
Delay Experienced by the End User:
This is the time taken to establish a certain service from the point when the user request the service
until the user start to receive information once the service is established. The delay experienced by a
streaming service user is the sum of GPRS attach time, PDP context activation and total setup times.
5
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
QoS Key Performance Indicators
•
Packet Delay Variation (Jitter): This is the delay variation of arrived packets.
•
Packet losses: The number of media packets (video/audio) lost during the streaming session.
•
Throughput: The number of received bits per unit time. Total throughput includes (media + headers)
throughputs.
•
Number and length of rebufferings: This measure is of great importance to the user because the
rebuffering operation can stop the audio/video playback. In this thesis only the rebuffering periods that
cause a pause in video/audio playback are taken into account.
•
Subjective Audio/Video quality: The following simple rating scale is defined through streaming QoS
measurements
• Excellent
• Good
• Fair
• Poor
• Bad
•
Lip synchronisation: This metrics is used to measure the synchronization between audio and video. The
metrics can be subjective and objective. The former can be based on the following rating scale.
•
subjective Lip synchronization
–
–
–
•
6
© NOKIA
In synch
Just noticeably out-of-synch
Unacceptably out-of-synch
The later can be calculated in the streaming client as the ratio between displayed frames to
decoded frames (Lip synchronization index)
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Environment description
•
7
The measurements were done in the NTN (Nokia Test Network), in Leppävaara area during weeks 45-52 ,
2004
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Measurements Tools
•
•
•
•
•
•
•
•
•
•
•
•
•
•
8
Nokia OSS system for logging cell load information
Nemo Outdoor
Global Positioning System (GPS)
Nokia 3G Test Mobile “ Nokia 7600 model”
Nokia Traffic Generator Tool
Helix Mobile Producer
Commview Packet Analyser
Ethereal Packet Analyser
Matlab Tool
Excel
Windows Grep
Nokia Internal Streaming Client
Nokia Internal Streaming Server
Helix Universal Server -Mobile
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Preliminary Streaming QoS Tests
Objectives: Preliminary evaluation of multimedia streaming over Nokia WCDMA Test Network with different
client –server streaming solutions.
Test Description:
Four tests were executed by using two different streaming client–server solutions, two tests for each solution.
The client-server streaming solutions are:
1.
Nokia Internal Streaming Client and Nokia Internal Streaming Server
2.
Nokia Internal Streaming Client and Helix Universal Server-Mobile
The test was done in stationary situation near to the WCDMA base station where the carrier to interference
ratio Ec/No > -6 dB. Two different RT PS RABs were used, 64/64 and 64/128 UL/DL. Multimedia files were
used with bit rates 51 and 115 kbps over 64/64 and 64/128 bearers respectively.
9
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Preliminary Streaming QoS Tests
Preliminary test 1:Multimedia streaming over 64/64 kbps UL/DL RT PS RAB, BLER target 1%
Environment: Nokia internal streaming client + Nokia internal streaming
The test results are presented in Table 1. Average video/audio packets sizes are 450 and 150 byte respectively
Table 1 Streaming performance over 64/64 RT RAB
Video
Audio
Subjective QoS (MOS)
Good
Good
Packet Loss Ratio, Number of
expected packets to be received
0.5%, 589
0.0%, 750
The results show a good performance of multimedia streaming when using Nokia Streaming Client –Server. The
MOS was “Good” for both audio and video.
10
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Preliminary Streaming QoS Tests
Preliminary test 2: Multimedia streaming over 64/128 kbps UL/DL RT PS RAB, BLER target 1%
Environment: Nokia internal streaming client + Nokia internal streaming
The test results are presented in Table 2. Average video/audio packets sizes are 381 and 313 byte respectively
Table 2 Streaming performance over 64/128 RT RAB
Video
Audio
Subjective QoS (MOS)
Fair
Fair
Packet Loss Ratio, Number of
expected packets to be received
1.9%, 2000
0.3%, 300
The results show a satisfactory performance of multimedia streaming when using Nokia Streaming Client –Server
solution. The MOS was “Fair” for both audio and video. Video packet losses were in acceptable level.
11
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Preliminary Streaming QoS Tests
Preliminary test 3: Multimedia streaming over 64/64 kbps UL/DL RT PS RAB, BLER target 1%
Environment: Nokia internal streaming client + Helix Server
The test results are presented in Table 3. Average video/audio packets sizes are 485 and 150 byte respectively.
The results show a slightly high video packet losses and low audio packet losses, high video packet losses explain
the poor perceived video quality
Streaming Server sending bit rate more
than guaranteed channel bit rate
Table 3 Streaming performance over 64/64RT RAB
Figure 1 Helix Server throughput over 64/64 RT bearer
12
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Video
Audio
Subjective QoS (MOS)
Poor
Fair
Packet Loss Ratio,
Number of expected
packets to be received
6.6%, 634
0.3%, 299
Preliminary Streaming QoS Tests
Preliminary test 4: Multimedia streaming over 64/128 kbps UL/DL RT PS RAB, BLER target 1%
Environment: Nokia internal streaming client + Helix Server
The test results are presented in Table 4. Average video/audio packets sizes are 333 and 150 byte respectively.
The results show a very high video/audio packet loss, which explain the poor perceived video/audio quality.
Streaming Server sending bit rate more
than guaranteed channel bit rate
Table 4 Streaming performance over 64/128RT RAB
Video
Audio
Subjective QoS (MOS)
Bad
Bad
Packet Loss Ratio,
Number of expected
packets to be received
29.6 %, 2136
31.1 %, 301
Figure 2 Helix Server throughput over 64/128 RT bearer
13
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Preliminary Streaming QoS Tests
•
The streaming performance over 64/64 and 64/128 UL/DL real time bearers was bad when using Helix
server.
•
The streaming performance over 64/64 and 64/128 UL/DL real time bearers was satisfactory when using
Nokia internal streaming server.
The tests had shown that streaming server was very often sending media packets with short delays and the
data rate exceeded the channel maximum bit rate. Data rate that exceed the maximum channel rate will be
discarded according to token bucket algorithm. The current implementation of token bucket algorithm is
done according to 3GPP specification [TS23.107 Annex B], which is not handling appropriately the burst
traffic situation. As a result of these findings the token bucket algorithm was modified to be able to tolerate
traffic burstiness conditions. This modification allows streaming class to use traffic shaping functionality,
which smooth the traffic pattern and decrease the burstiness of the network.
All the measurements that follow were done with the modified token bucket algorithm except SHO
measurements.
14
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Real Time Bearers in Unloaded Cell
Objectives:
To verify the performance of multimedia streaming over Nokia WCDMA Test Network in a stationary situation
Test 1: Multimedia streaming over 64/64 kbps UL/DL RT PS, BLER target 1%
Environment: Nokia internal streaming client + Helix Server
AT command used :at+cgeqreq=1,1,64,64,64,64,0,1500,"1E3",,0,1000
Total number of video and audio packets sent by streaming server is 589 and 750 respectively. Average video
and packet sizes are 485 and 150 byte respectively.
Table 5 Call setup QoS KPIs over 64/64 streaming bearer, BLER 1%
Trial
Setup time
Buffering Time
Total Setup time
msec
msec
msec
1
6539
5017
11556
2
6498
5017
11515
3
6522
5057
11579
The end user will experience the following delays before start to see the movie
e.g in trial 1 the end user experience the following delays
Attach time+PDP context time + setup time + buffering time = 6.7+6.53+5=18.23 sec
See GPRS attach and PDP context measurements.
15
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Real Time Bearers in Unloaded Cell
Table 6 End-to-End and objective/subjective video/audio QoS KPIs over 64/64 streaming bearer, BLER target 1%
Trial
Average
Throughput
PLR %
PLR%
audio
video
kbps
Packet Delay
video
Packet Delay
Audio
Min,max,avg
Min,max,avg
msec
msec
MOS
MOS
video
audio
Lip
Sync
Subjective lip
synchronizatio
n
index
Number and
Length of
rebufferings
(msec)
1
58.29
0.33%
1.26%
0,521,96.7
30,671,200.5
Good
Good
0.99
In synch
0/0
2
57.15
1.67%
2.05%
0,333,97
10,681,204.6
Fair
Good
0.69
In synch
0/0
3
57.99
1.67%
0.33%
0,301,96
40,681,204.2
Good
Good
0.85
In synch
0/0
Total throughput kbit/s (1 sec. window)
70
60
50
40
kbit/s
Figure 3 shows the total throughput
(media + header throughputs) received by the streaming
client. The received throughput is within the channel
bandwidth in most of the times.
Measured
Reference
30
20
10
0
1
11
21
31
41
51
sec.
Figure 3 Streaming client total throughput over 64/64 RT bearer
16
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Real Time Bearers in Unloaded Cell
Objectives:
To verify the performance of multimedia streaming over Nokia WCDMA Test Network in a stationary situation
Test 2: Multimedia streaming over 64/128 kbps UL/DL RT PS, BLER target 1%
Environment: Nokia internal streaming client + Helix Server
AT command used :at+cgeqreq=1,1,64,128,64,128,0,1500,"1E3",,0,1000
Total number of video and audio packets sent by streaming server is 2136 and 310 respectively. Average
video and packet sizes are 333 and 150 byte respectively.
Table 7 Call setup QoS KPIs over 64/128 RT streaming bearer, BLER 1%
Trial
17
© NOKIA
Setup time
Buffering Time
Total Setup time
msec
msec
msec
1
11141
5017
16158
2
7048
5017
12065
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Real Time Bearers in Unloaded Cell
Table 8 End-to-End and objective/subjective video/audio QoS KPIs over 64/128 streaming bearer, BLER target 1%
Trial
Average
Throughput
kbps
PLR
%
PLR%
video
audio
Packet Delay
video
Packet Delay
Audio
Min,max,avg
Min,max,avg
msec
msec
MOS
MOS
video
audio
Lip
Sync
Subjective
lip
synchronization
index
Number and
Length of
rebufferings
(msec)
1
119.44
0.3%
0.33 %
0,171,28.4
10,621,202
Good
Good
0.57
Just
noticeably
out of synch
0/0
2
119.32
0.3%
0.23%
0,160,28
20,601,201
Good
Good
0.97
In Synch
0/0
Packet losses are in acceptable level
Subjective QoS was good
Lip sync index shows a satisfactory values
No pause and rebufferings occurred during the
streaming session.
The received throughput is within the channel bandwidth
in most of the times.
Figure 4 Streaming client total throughput over 64/128 RT bearer
18
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Real Time Bearers in Unloaded Cell
Objectives:
To verify the performance of multimedia streaming over Nokia WCDMA Test Network in a stationary situation
Test 3: Multimedia streaming over 64/256 kbps UL/DL RT PS, BLER target 1%
Environment: Nokia internal streaming client + Helix Server
AT command used :at+cgeqreq=1,1,64,256,64,256,0,1500,"1E3",,0,1000
Total number of video and audio packets sent by streaming server is 4066 and 297 respectively. Average
video and packet sizes are 341 and 321 byte respectively.
Table 9 Call setup QoS KPIs over 64/256 streaming bearer, BLER 1 %
Trial
19
© NOKIA
Setup time
Buffering Time
Total Setup time
msec
msec
msec
1
10006
5017
15023
2
9020
5017
14037
3
10851
5017
15868
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Real Time Bearers in Unloaded Cell
Table 10 End-to-End and objective/subjective video/audio QoS KPIs over 64/256 streaming bearer, BLER target 1%
Trial
Average
Throughput
PLR %
PLR%
audio
video
kbps
Packet Delay
video
Packet Delay
Audio
Min,max,avg
Min,max,avg
msec
msec
MOS
MOS
audio
video
Lip
Sync
Subjective lip
synchronization
index
Number and
length of
rebufferings
msec
1
203.73
0.9%
0.93 %
0,821,16.9
10,991,207
Fair
Poor
0.46
Unacceptably
out-of-synch
5 / 5860
2
177
1%
1%
0,4707,17
10,1192,206
Fair
Bad
0.34
Unacceptably
out-of-synch
11 / 5947
3
187
1.3%
0.8%
0,3345,17.6
0,1192,209
Fair
Bad
0.38
Unacceptably
out-of-synch
5 / 5860
Total throughput kbit/s (1 sec. window)
400
350
300
250
kbit/s
Figure 5 shows the total throughput received by the
streaming client. The total throughput was exceeding
the channel bandwidth in most of the times .
Subjective video/audio quality was not totally satisfactory
Lip sync index shows a very low values which indicates
that the video and audio are out of sync
Many rebufferings occurred during the video.
Measured
200
Reference
150
100
50
0
1
11
21
31
41
51
sec.
Figure 5 Streaming client total throughput over 64/128 RT bearer
20
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell
Objectives:
Total number of video and audio packets sent by streaming server is 2136 and 300 To verify the
performance of multimedia streaming over NRT bearer in a stationary situation
Test 4: Multimedia streaming over 64/64 kbps UL/DL NRT PS, BLER target 1%
Environment: Nokia internal streaming client + Helix Server
respectively. Average video and packet sizes are 333 and 321 byte respectively.
Table 11 Call setup QoS KPIs over 64/64 NRT streaming bearer, BLER 1 %
Trial
21
© NOKIA
Setup Time
Buffering Time
Total Setup time
msec
msec
msec
1
6839
5017
11846
2
6778
5017
11795
3
6922
5057
11979
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell
Table 12 End-to-End and objective/subjective video/audio QoS KPIs over 64/64 NRT bearer, BLER target 1%
Trial
Average
Throughput
PLR %
PLR%
audio
video
kbps
Packet
Delay video
Packet Delay
Audio
Min,max,avg
Min,max,avg
msec
msec
MOS
video
audio
Lip
Sync
Subjective
lip
synchronization
index
Number and
Length of
rebufferings
(msec)
1
58.29
0.0%
0.0 %
0,555,96.7
30,671,200
Good
Good
0.86
In Sync
0/0
2
57.15
0.0%
0.0%
0,340,97
10,681,204
Good
Good
0.76
In Sync
0/0
3
57.99
0.0%
0.0%
0,370,96
40,680,203
Good
Good
0.82
In Sync
0/0
No Packet losses
Subjective video and audio QoS was good
Lip sync index shows a good values
No pause and rebuffering occurred during the
streaming session.
22
MOS
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell
Objectives:
To verify the performance of multimedia streaming over NRT bearer in a stationary situation
Test 5: Multimedia streaming over 64/128 kbps UL/DL NRT PS, BLER target 1%
Environment: Nokia internal streaming client + Helix Server
Total number of video and audio packets sent by streaming server is 2136 and 300 respectively. Average
video and packet sizes are 333 and 321 byte respectively.
Table 13 Call setup QoS KPIs over 64/128 NRT streaming bearer, BLER 1 %
Trial
23
© NOKIA
Setup Time
Buffering Time
Total Setup time
msec
msec
msec
1
10230
5017
15247
2
8921
13938
11338
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell
Table 14 End-to-End and objective/subjective video/audio QoS KPIs over 64/128 NRT bearer, BLER target 1%
Trial
Average
Throughput
PLR %
audio
kbps
PLR
%
Packet Delay
video
Packet Delay
Audio
video
Min,max,avg
Min,max,avg
msec
msec
MOS
MOS
video
audio
Lip
Sync
Subjective
lip
synchronization
index
Number and
length of
rebufferings
msec
1
119.98
0.0%
0.0 %
0,231,28.3
20,590,200
Good
Good
0.62
In Sync
0/0
2
119.96
0.0%
0.0%
0,230,28.4
10,661,202
Good
Good
0.81
In Sync
0/0
Total throughput kbit/s (1 sec. window)
No Packet losses
Subjective video and audio QoS was good
Lip sync index shows a good values
No pause and rebuffering occurred during the
streaming session.
The received throughput is exceeding the channel
Bandwidth sometimes
250
200
kbit/s
150
Measured
Reference
100
50
0
1
11
21
31
41
51
sec.
Figure 6 Streaming client total throughput over 64/128 NRT bearer
24
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell
Objectives:
To verify the performance of multimedia streaming over NRT bearer in a stationary situation
Test 6: Multimedia streaming over 64/384 kbps UL/DL NRT PS, BLER target 1%
Environment: Nokia internal streaming client + Helix Server
Total number of video and audio packets sent by streaming server is 4066 and 301 respectively. Average
video and packet sizes are 333 and 321 byte respectively.
Table 15 Call setup QoS KPIs over 64/384 NRT streaming bearer, BLER 1 %
Trial
25
© NOKIA
Setup Time
Buffering Time
Total setup time
msec
msec
msec
1
9813
5018
14831
2
12071
5017
17071
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming QoS Measurements over Non Real Time Bearers in Unloaded Cell
Table 16 End-to-End and objective/subjective video/audio QoS KPIs over 64/384 NRT bearer, BLER target 1%
Trial
Average
Throughput
PLR %
PLR%
audio
video
kbps
Packet Delay
video
Packet Delay
Audio
Min,max,avg
Min,max,avg
msec
msec
MOS
MOS
audio
video
Lip
Sync
Subjective lip
synchronization
index
Number and
length of
rebufferings
msec
1
213.94
0.8 %
1.1 %
0,390,15.3
0,1041,204
Good
Fair
0.45
Just noticeably
out-of-synch
2 / 5743
2
219.4
0.0%
0.1%
0,491,14.9
0,931,203
Good
Fair
0.5
Just noticeably
out-of-synch
2/ 5944
Total throughput kbit/s (1 sec. window)
400
350
300
250
kbit/s
Packet losses ~ 1%
Subjective audio/video QoS was satisfactory
Lip sync index shows a relatively low values
The received throughput is exceeding the channel
Bandwidth very often
Measured
200
Reference
150
100
50
0
1
11
21
31
41
51
sec.
Figure 7 Streaming client total throughput over 64/384 NRT bearer
26
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming in Highly Loaded Cell With Conversational Class Users
Objective: To verify that the guaranteed bit rate of a streaming class user is kept in a highly loaded cell.
Test Description:
The test was executed by starting a single video streaming session with a predefined data rate and loading
the cell with conversational class users (voice calls). The throughput of streaming user was monitored.
Environment:
Increasing common channels power by 11 dB
27
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming in Highly Loaded Cell With Conversational Class Users
Test 7: Streaming over 64/64 RT bearer
DL BLER
High BLER
Throughput decreased
significantly due to packet
losses
Throughput PPP Layer
CPICH RSCP
UE Tx Pwr
CPICH Ec/No
Total # of send packets =1389
Total # of packets lost = 15
Packet losses ~1%
28
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming in Highly Loaded Cell With Conversational Class Users
Test 7: Streaming over 64/64 RT bearer
UL RT Load
PtxTotal DL=~ Ptxtarget DL
29
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
New AMR calls are
not admitted after
33 existing AMR calls
Streaming in Highly Loaded Cell With Conversational Class Users
Test 8: Streaming over 64/128 RT bearer
Throughput is not affected by cell congestion
DL BLER
Throughput PPP Layer
CPICH RSCP
UE Tx Pwr
CPICH Ec/No
Total # of send packets =8118
Total # of packets lost = 36
Packet losses 0.4%
30
© NOKIA
New AMR calls are
not admitted after
37 existing AMR calls
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming in Highly Loaded Cell With Conversational Class Users
Test 9: Streaming over 64/256 RT bearer
DL BLER
Throughput PPP Layer
CPICH RSCP
UE TX Power
CPICH EcNo
Total # of send packets =8247
Total # of packets lost = 34
Packet losses 0.4%
31
© NOKIA
New AMR calls are
not admitted after
37 existing AMR calls
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming Performance Measurement in Mobile Situation
•
Objective: To verify the performance of multimedia streaming over real time bearer during soft handover in
Nokia WCDMA Test Network.
The description of tests is as follow:
•
Two tests were executed over two streaming class bearers, one test per bearer. The bearer bit rates are
64/64 and 64/128 kbps UL/DL. BLER target value was set to 1% for all tests. Multimedia files were used with
bit rates 103 and 213 kbps over 64/128 and 64/256 RT PS RABs respectively. The tests were conducted by
using Nokia internal streaming client and Nokia streaming server. Streaming performance was verified
during soft handover. The carrier to interference ration was “Ec/No > -6” dB.
Every test is repeated 2-3 times to
ensure reliability.
This test was executed with original
token bucket algorithm
Softer handover
Soft handover
32
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming Performance Measurement in Mobile Situation
Streaming over 64/128 RT bearer
Objectives:
To verify the performance of multimedia streaming during SHO
Test 10: Multimedia streaming during SHO over 64/128 kbps UL/DL RT PS, BLER target 1%
Environment: Nokia internal streaming client + Nokia internal streaming server
AT command used :at+cgeqreq=1,1,64,128,64,128,0,1500,"1E3",,0,1000
Total number of video and audio packets sent by streaming server is 1880 and 296 respectively. Average
video and packet sizes are 382 and 313 byte respectively.
Table 17 Call setup QoS KPIs over 64/128 streaming bearer, BLER 1 %
Trial
Setup time
Buffering Time
msec
msec
Total Setup
time
msec
33
© NOKIA
1
4196
5017
9213
2
4126
5017
9143
2
3935
5017
8952
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming Performance Measurement in Mobile Situation
Streaming over 64/128 RT bearer
Table 18 End-to-End and objective/subjective video/audio QoS KPIs over 64/128 RT bearer during SHO, BLER target 1%
Trial
Average
Throughput
kbps
PLR
%
PLR%
video
audio
Packet
Delay video
Packet Delay
Audio
Min,max,avg
Min,max,avg
msec
msec
MOS
MOS
video
audio
Lip
Sync
Subjective
lip
synchronization
index
Number and
Length of
rebufferings
(msec)
1
112.8
6%
7.34%
0,221,33
10,651,212
Fair
Good
0.58
Just noticeably
out-of-synch
0/0
2
112.8
5.4%
7.5 %
0,341,34
60,631,210
Fair
Good
0.56
Just noticeably
out-of-synch
0/0
3
112.9
7%
7.3%
0,251,33
20,570,214
Fair
Good
0.55
Just noticeably
out-of-synch
0/0
Total throughput kbit/s (1 sec. window)
Video packet losses are high due to token packet
algorithm problem
Lip sync index shows a satisfactory values
No rebufferings occurred during the streaming
session.
140
120
100
kbit/s
80
Measured
Reference
60
40
20
0
1
11
21
31
41
51
sec.
Figure 8 Streaming client total throughput over 64/128 RT bearer
34
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming Performance Measurement in Mobile Situation
Streaming over 64/128 RT bearer
Soft handover takes place between different base stations sites within one RNC (intra-RNC soft handover) or
within separate controlled RNCs (inter-RNC soft handover). In softer handover the UE is controlled by at least
two cells belonging to the same base station site.
Softer HO
35
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Soft HO
Streaming Performance Measurement in Mobile Situation
Streaming over 64/256 RT bearer
Objectives:
To verify the performance of multimedia streaming during SHO
Test 11: Multimedia streaming during SHO over 64/256 kbps UL/DL RT PS, BLER target 1%
Environment: Nokia internal streaming client + Nokia internal streaming server
AT command used :at+cgeqreq=1,1,64,256,64,256,0,1500,"1E3",,0,1000
Total number of video and audio packets sent by streaming server is 4066 and 297 respectively. Average
video and packet sizes are 341 and 321 byte respectively.
Table 19 Call setup QoS KPIs over 64/256 streaming bearer, BLER 1 %
Trial
36
© NOKIA
Setup time
Buffering Time
Total Setup time
msec
msec
msec
1
3675
5017
8692
2
3856
5017
8873
2
3846
5017
8863
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming Performance Measurement in Mobile Situation
Streaming over 64/256 RT bearer
Table 20 End-to-End and objective/subjective video/audio QoS KPIs over 64/256 RT bearer during SHO, BLER target 1%
Trial
Average
Throughput
PLR %
PLR%
audio
video
kbps
Packet Delay
video
Packet Delay
Audio
Min,max,avg
Min,max,avg
msec
msec
MOS
MOS
audio
video
Lip
Sync
Subjective lip
synchronization
Number and
length of
rebufferings
msec
index
1
220
2.3%
7.25%
0,180,22
60,460,204
Good
Fair
0.56
Just noticeably
out-of-synch
0/0
2
221
1.3%
6.95%
0,170,21
10,445,202
Good
Fair
0.56
Just noticeably
out-of-synch
0/0
3
220
3.3%
7.05%
0,330,21
30,511,207
Good
Fair
0.55
Just noticeably
out-of-synch
0/0
Total throughput kbit/s (1 sec. window)
300
Video packet losses are high due to token packet
algorithm problem
Subjective audio/video QoS was satisfactory
Lip sync index shows satisfactory values
No rebufferings occurred during the streaming
session.
250
kbit/s
200
Measured
150
Reference
100
50
0
1
11
21
31
41
51
sec.
Figure 9 Streaming client total throughput over 64/256 RT bearer
37
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Streaming Performance Measurement in Mobile Situation
Streaming over 64/256 RT bearer
Softer HO
38
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Soft HO
GPRS Attach and PDP Context Activation Times
Objectives: To measure the GPRS attach and PDP context activation times over streaming class bearers.
Three different RAB bit rates are used, 64/64, 64/128, and 64/256 kbps UL/DL. BLER target value was 1% for
all tests. Every test was repeated 15 times to ensure reliability.
Attach Times were calculated as the difference time between ATTACH_REQ and ATTACH_COMPLETE
messages.
PDP context activation time was calculated as a time difference between PDP_context_create_req and
PDP_context_activate_ind messages
Table 33 shows that average PDP context + GPRS attach time for 64/64, 64/128, and 64/256 are 6.71, 7.09,
and 7.28 seconds.
Table 21 Average GPRS attach+ PDP Context Activation.
Real Time Bearer
PDP context + GPRS attach
Average Time (sec)
39
© NOKIA
64/64
6.71
64/128
7.09
64/256
7.28
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Summary & Conclusions
•
The preliminary evaluation tests using Nokia internal streaming server had shown a good performance of
multimedia streaming over 64/64 UL/DL real time bearer and a satisfactory performance over 64/128 UL/DL
real time bearer.
•
The preliminary evaluation tests using Helix streaming server had shown a bad performance of multimedia
streaming over 64/64 and 64/128 UL/DL real time bearers.
The following conclusions are drawn from the preliminary evaluation tests:
40
•
Nokia streaming server has shown a better performance than Helix streaming server.
•
The bad performance of multimedia streaming over streaming class was due to the large number of
discarded packets by the token bucket algorithm. The current implementation of token bucket algorithm,
which is according to 3GPP specification was not suitable to control the traffic flow of streaming servers.
•
The results obtained after token bucket modification have shown a good performance of multimedia
streaming over 64/64 and 64/128 real time bearers and the performance over 64/256 UL/DL RT bearer was
not totally satisfactory.
•
The performance of video streaming over 64/64 and 64/128 NRT bearers was good. Video and audio quality
was good and video/audio packet losses were 0.0% in all trials.
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
Summary & Conclusions
•
The performance of video streaming over 64/384 UL/DL NRT bearer was satisfactory.
•
The performance of multimedia streaming over 64/128 and 64/256 UL/DL RT during SHO was satisfactory.
(original token bucket algorithm + Nokia streaming server)
The following conclusions are drawn from the streaming verification tests after token bucket algorithm
modifications.
•
41
•
The performance of video streaming over real time bearer was significantly improved after token bucket
algorithm modification.
•
Multimedia streaming over NRT bearers in unloaded cell show a slightly better performance than
streaming over RT bearers.
The streaming service user will experience a delay of ~20 seconds before the media contents starts to be
displayed in the streaming client. This delay includes attach, context, and total setup times.
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005
References
[Hol] Harri Holma and Antti Toskala, ”’WCDMA for UMTS Radio Access For Third Generation Mobile
Communications”, Second Edition, John Wiley & Sons, LTD, 2002
[Lai] Jaana Laiho, Achim Wacker and Tomas Novosad, “ Radio Network Planning and Optimisation for UMTS”,
John Wiley & Sons, LTD, 2002, 484 pages.
[22105] 3GGP TS 22.105 Service aspects; Services and Service Capabilities, version 4.1.0, January 2001, 31
pages.
[23107] 3GPP Technical Specification 23.107: QoS concept and Architecture, version 4.6.0, December 2002, 38
pages
42
© NOKIA
Streaming QoS/Musa Aldarawsheh , April 8th 2005