Introduction to UMTS RAN KPI
GSM-to-UMTS Training Series_V1.0
Change History
Date
Revision Version
Description
Author
2008-12-13
1.0
Draft completed.
Zang Liang
2009-01-08
1.1
The remarks of basic concept are added in
pages 4, 13, 19, 28, and 43. The remarks in
page 26 are modified.
Guo Hao
Page 2
Objectives
•
•
•
Understand the definition of the RAN KPI
Familiar with basic KPI test method
Know how to analyze the network performance
through the KPI
 After learning this course, we can know:
 Basic items for network acceptance test
 DT and CQT test methods of the KPI and traffic
statistics analysis
 Simple signaling analysis
Page 3
Foreword: UMTS&GSM KPI List
UMTS KPI
GSM KPI
G&U KPI Difference
Coverage rate
Coverage rate
CPICH Ec/Io and RSCP thresholds are set in the WCDMA
network
RXLEV threshold is set in the GSM network.
SHO rate
None
SHO rate is a specific KPI for the CDMA service, which is
closely related to the resource usage.
RRC connection setup
success rate
Success rate of immediate
assignments
When the RRC signaling is established in the WCDMA
network, it cannot differentiate the PS service and the CS
service.
RAB connection setup
success rate
Success rate of assignments
The RAB connection setup success rate involves the Voice,
VP and PS services, and is similar to the success rate of the
TCH assignment and TBF setup success rate.
Call drop rate
Call drop rate
The service type of the WCDMA network is different from
that of the GSM network.
Delay
Delay
It is tested by the drive test (DT) and call quality test (CQT),
The KPIs related to the delay in the WCDMA network are
the same as the KPIs related to the GPRS service in the
GSM network.
Handover success rate
Handover success rate
The classification of the KPIs in the WCDMA network and
the GSM network is different.
Page 4
Contents
Introduction to KPI
Test Method of KPI
RAN KPI Counters
Page 5
Introduction to KPI
• Definition of KPI
 Key Performance Indicator
• Management of KPI
 RNC-level, cell-level, and customized level
• Function of KPI
 Define the network performance counters
 Monitor the network performance
 Locate and rectify the network faults
 Analyze the network trend
• Purpose of KPI Test
 The verification and enhancement of the network optimization. That is, evaluate
the network performance through the network KPI test, solve the network
problem, continue to optimize the network to achieve the objective of building a
refined network.
Page 6
Introduction to
KPI
End-to-End Network Performance Evaluation System
Other PLMN
Core Network
Service Network
PSTN
Internet
Radio/Transmissi
User
Service
Core Network
on
Equipmen
Network
Performance
Network
t
Performance
Performance
Performance
End-to-End Service Performance
External
Network
Performance
End-to-End Quality of Experience
UMTS network performance evaluation system is used to map the service quality onto the end-to-end counters,
which comprehensively indicate the UE performance, radio network performance, transport network performance,
core network performance, and service network performance, and external network performance.
Page 7
Introduction to KPI
Relation of QoE, KQI, and KPI
Quality of Experience
(QoE)
Availability
Key Quality
Indicator (KQI)
Key
Performance
Indicator
(KPI)
Accuracy
Speed
Voice, Video Phone, MMS, WAP, Email, FTP, HTTP…
AMR, CSD64k, CS Streaming, PS Steaming, BE64k/128k/384k, HSDPA
Accessibility
Retain ability
Integrity
Mobility
Raw Counters, Drive Test, …
Page 8
….
Coverage
Availability
Introduction to KPI
Customer:
As a customer, I do not care about KPIs, only
in the hope of obtaining good and rich
services.
Operator:
Our ultimate purpose is to meet increasing
requirements of network subscribers and
provide wireless networks with perfect user
experience.

As a channel between subscribers and the network, subscribers experience the network
quality through services

"Focus on User Experience " is a prerequisite and feature of a perfect network KPI system.
Page 9
Introduction to
KPI
Huawei RAN KPIs
Coverage AccessabilityRetainability Service Integrity Mobility
• Coverage
rate
• RRC connection
setup success rate
• SHO rate
• RAB connection
setup success rate
• Call setup
success rate
• Pilot
pollution
ratio
• PDP activation
success rate
• Call completion
rate of the calling
party (voice, video,
and SMS)
• Call completion
rate of the called
party (voice, video,
and SMS)
• Call drop
rate (voice,
video, and
PS services)
• Mean opinion
score (MOS)
• Soft handover
success rate
• Video call
(synchronization
performance of
voice and image
• Intra-frequency
hard handover
success rate
• Uplink/downlink
BLER (Voice,
Video Phone)
• Average
throughput of PS
services
(64K/128K/384
K/HSPA)
• Inter-frequency
hard handover
success rate
• Inter-RAT
handover
success rate
Delay
Availability
• Call setup delay
• Worst cell rate
• PDP activation
delay
• Admission
rejection rate
• Point-to-point
user plane delay
• Paging
congestion rate
• Ping delay
• Downlink
transmission
resource usage
rate
• Service
disruption delay
• Flow delay
Page 10
• Congested cell
rate
Contents
Introduction to KPI
Test Method of KPI
RAN KPI Counters
Page 11
Test Method of KPI
•
Test Method

•
Prerequisite




•
The system works normally and stably.
The test route must meet the following conditions:
① Test requirements of coverage rate: PCPICH RSCP >= -95dBm &
Ec/Io >= -12dB
② Test requirements of other counters: PCPICH RSCP >= -90dBm &
Ec/Io >= -11dB
Basic operation of the network optimization is complete.
The downlink analog load cannot exceed the planned load (optional).
Test Tool and Software


Page 12
Traffic statistics, drive test (DT), and call quality test (CQT)
Mobile phone, frequency scanner, GPS, and laptop computer
LMT, M2000, Probe, Assistant, and Nastar

DT——Drive Test
Classification of KPI
Test Methods
 Evaluate the overall network performance after network optimization.
− For example, mileage call drop ratio, Voice/VP call access rate/call drop rate of the calling party
or called party, soft handover success rate, and Inter-RAT CS/PS HO success rate

ST——Static/Stationary Test
 Evaluate the processing capability and speed of each network element.
− Such as voice/VP call completion rate/call drop rate of the calling party or called party, PDP
activation success rate/delay, uplink/downlink throughput, MOS, UE2UE/PSTN Voice/VP call
setup delay/user plane delay, and Ping delay

Stat.——Statistics
 Evaluate the overall network quality.
− Such as SHO overhead, uplink interference cell rate, RRC/RAB connection setup success rate,
Voice/VP/PS call drop rate, SHO success rate, Inter-RAT CS/PS HO success rate, worst cell rate,
admission rejection rate, and congested cell rate
Page 13
Test Method of KPI
•
Advantages and Disadvantages of DT/ST and stat.
–
Complementary to each other, DT/ST and Stat. must be flexibly used at different network
phases and under different network conditions.
Pro
Con
1. Close to actual user experience.
DT/ST
2. Unrestricted to the traffic.
1. Restricted to network conditions.
3. Easy to locate the fault.
2. Unsuitable for long-time observation.
4. Easy to test the specific network
performance
3. Often affected by subjective and human factors.
1. Evaluating overall network performance
Stat.
2. Suitable for long-time observation
3. Less affected by subjective factors and
human factors
–
Page 14
1. Unnecessarily close to actual user experience
2. Restricted to the traffic volume
3. Difficult to locate the fault
4. Difficult to adjust test items to test network
performance
Another prerequisite of the KPI of the high-quality network is that all the KPIs of
DT/ST/Stat must be covered and all kinds of counters must be complementary to each
other.
Test Method of KPI
•
Life Cycle of DT/ST/Stat.
Network lifecycle
Contract
Requirements
for KPIs
Planning
Implementation
Optimization
Cluster
Optimization by
•Field Test
•Field Test KPIs KPIs (Drive
Test/
(Drive Test/
Stationary Test) Stationary
Test)
for PAC
•Statistic KPIs
for FAC
Page 15
Ongoing
Optimization
Commercial
Launch
Network
Expansion
FAC1
•Stat.:
•RRC/RAB SSR
•CDR
•SHO SSR
•...
PAC
•ST:
•Call Setup Delay
•PDP Context
Activation Delay
•Ping Delay
•...
PAC
•DT:
•CDR
•CSSR
•MOS
•...
Ongoing Optimization
by
•Field Test KPIs
(Drive
Test/Stationary
Test)
FAC2
•Stat.:
•RRC/RAB SSR
•CDR
•SHO SSR
•...
Contents
Introduction to KPI
Test Method of KPI
RAN KPI Counters
Page 16
RAN KPI List
KPI Type
KPI Name
KPI Name
Mileage call drop ratio
Soft handover success rate
Coverage rate
Intra-frequency handover success rate
SHO rate
Coverage
KPI Type
Mobility
Inter-frequency handover success rate
SHO overhead
Inter-RAT handover success rate
Pilot pollution ratio
......
Uplink Interference cell rate
Call setup delay
......
PDP activation delay
......
UE2UE user plane delay
RRC connection setup success rate
UE2PSTN user plane delay
Delay
Accessibility
Retainability
RAB connection setup success rate
Ping delay
Call completion rate of the calling party
Service disruption delay
Call completion rate of the called party
Flow delay
PDP activation success
......
......
Worst cell rate
Call drop rate
Admission rejection rate
......
Rate of relatively high uplink BLERs
Availability
Congested Cell rate
......
Rate of relatively high downlink BLERs
Average downlink throughput
Service integrity
Average uplink throughput
Mean Opinion Score (MOS)
......
Page 17
Remarks:
1. All the KPIs have two modes: Stat. mode and DT/ST mode.
2. According to the services, some KPIs have sub-item.
3. According to the calling part or the called party, some KPIs have
sub-item.
RAN KPI Counters
Coverage
Accessibility
Retainability
Delay
Mobility
Page 18
Coverage Analysis
• KPI (DT/CQT)
Coverage Rate
At the early phase of the network deployment, the coverage rate is an important
factor in the network planning and optimization.
• KPI (stat.)
SHO Rate and SHO Overhead
The impact of the overlapped coverage on the GSM network is not obvious. For
the WCDMA network, however, it may decrease the network capacity and
deteriorate the coverage quality (pilot pollution).
Page 19
Coverage
Coverage rate
•
•
•
•
Page 20
Test Method: DT
 Coverage rate＝A1/A2*100%
 A1: length of valid test route (km) (CPICH Ec/Io≥-12dB & RSCP≥-95dBm)
 A2: total length of test route
 The coverage rate is calculated after the distance binning of 5 m is performed.
Main Influencing Factors
 NodeB CPICH transmit power
 NodeB distribution condition
 Environmental factor
 Total interference level
Test Conditions
– Perform the test on the customer-specified route.
– The area without coverage should not be included in the customer-specified route.
Reference value of class A network: coverage rate >=96％
Coverage
SHO rate
•
Test Method: DT or stat. (evaluate the proportion of valid overlapped area)

SHO rate = (Number of samples that 2 radio links in active set + Number of samples that 3
radio links in active set)/Total number of samples x 100%
SHO_FACTOR_UE = (A2 + A3) / (A1 + A2 + A3)
•
•
Page 21

A1: 1 radio links in active set

A2: 2 radio links in active set

A3: 3 radio links in active set
Main Influencing Factors

Soft handover parameter

Site density and transmit power

Average antenna height, radio propagation environment, antenna azimuth, and antenna tilt
angle of the BTS

Traffic load of the sector (mainly specifies the load of the Um interface, and is not related to
the actual capacity of the network)

Traffic distribution and MS distribution
Reference value of class A network: SHO rate <=40%
Coverage－SHO Overhead
SHO overhead
SHO Overhead＝{(ACTIVESET_1_UE_MEAN + ACTIVESET_2_2SOFTER_UE_MEAN*2 + ACTIVESET_2_2SOFT_UE_MEAN*2 +
ACTIVESET_3_3SOFTER_UE_MEAN*3 + ACTIVESET_3_3SOFT_UE_MEAN*3 + ACTIVESET_3_2SOFTER_UE_MEAN*3) /
(ACTIVESET_1_UE_MEAN + ACTIVESET_2_2SOFTER_UE_MEAN + ACTIVESET_2_2SOFT_UE_MEAN +
ACTIVESET_3_3SOFTER_UE_MEAN + ACTIVESET_3_3SOFT_UE_MEAN + ACTIVESET_3_2SOFTER_UE_MEAN)} -1
Note : ACTIVESET_1_UE_MEAN: average number of UEs configured with one radio links.
ACTIVESET_2_2SOFTER_UE_MEAN: average number of UEs configured with two radio links
ACTIVESET_2_2SOFT_UE_MEAN: average number of UEs configured with two radio links that are not combined
ACTIVESET_3_3SOFTER_UE_MEAN: average number of UEs configured with three radio links that are not combined.
ACTIVESET_3_3SOFT_UE_MEAN: average number of UEs configured with three radio links that are not combined.
ACTIVESET_3_2SOFTER_UE_MEAN: average number of UEs configured with three radio links that two links are combined.

Reference value of class A network: SHO overhead <=45%

Formula for calculating SHO overhead:
SHO_Ratio = 100*{[VS.SHO.AS.1 +
(VS.SHO.AS.2Softer + VS.SHO.AS.2Soft)*2+
(VS.SHO.AS.3Softer+
VS.SHO.AS.3Soft+VS.SHO.AS.3Soft2Softer)*3)]/
( VS.SHO.AS.1 + VS.SHO.AS.2Softer +
VS.SHO.AS.2Soft + VS.SHO.AS.3Softer +
VS.SHO.AS.3Soft + VS.SHO.AS.3Soft2Softer )-1}

SHO overhead
In the Nastar software, the counter defined by the SHO
overhead is as follows:
SHO Overhead＝VS.SHO.Factor.RL
Page 22
RAN KPI Counter
Coverage
Accessibility
Retainability
Delay
Mobility
Page 23
•
KPIs (stat.):
Accessibility
Analysis
RRC connection setup success rate and RAB connection setup success rate
These counters are mapped onto the success rate of immediate assignments and success
rate of assignments in the GSM network. Similar to the analysis of the GSM accessibility,
you can focus on these counters in the daily KPI monitor and network optimization. There is
no KPI about the congestion in the WCDMA network. Therefore, you need to analyze the
network congestion through these counters.
•
KPIs (DT/CQT)
Call completion rate of the calling/called party and PDP activation success rate
The test requirements of KPIs in the WCDMA network are basically the same as the test
requirements of KPIs in the GSM network. The service type in the WCDMA test is increased,
so the classification of the tested KPIs is also increased. Like the KPIs of GPRS/EDGE services,
the KPIs of PS services are classified according to different rates. For the CS services in the
WCDMA network, the VP service test is added.
Page 24
RRC Connection Setup
Success Rate
•
Test Method: stat.




Stat. Number of RRC Connection Request in Point A1 of Figure1
Stat. Number of RRC Connection Setup in Point B of Figure1
Stat. Number of RRC Connection Setup complete in Point C of Figure1
Stat. Number of RRC Connection Reject in point A of Figure2
UE
RNC
RRC CONNECTION REQUEST
A1
UE
RNC
A2
RRC CONNECTION REQUEST
RRC CONNECTION SETUP
B
RRC CONNECTION REJECT
RRC CONNECTION SETUP COMPLETE
A
C
F1:successful RRC connection setup procedure
F2: RRC connection reject procedure
Page 25
•
RRC Connection Setup
Success Rate
RRC Setup KPI (reference value of class A network: RRC connection setup success rate >=98.5%)

RRC Connection Setup Success Rate = RRC connection setup success time / RRC connection request time received by RNC
× 100％
–
•
Note：here the RRC connection setup request time is the request time of initiating the service
Calculation Formula of RRC Connection Setup Success Rate:
–
RRC Connection Setup Success Rate ＝100 *∑(VS.RRC.SuccConnEstab.OrgConv + VS.RRC.SuccConnEstab.OrgStr +
VS.RRC.SuccConnEstab.OrgInt + VS.RRC.SuccConnEstab.OrgBkg + VS.RRC.SuccConnEstab.OrgSubs +
VS.RRC.SuccConnEstab.TrmConv + VS.RRC.SuccConnEstab.TrmStr + VS.RRC.SuccConnEstab.TrmInt +
VS.RRC.SuccConnEstab.TrmBkg + VS.RRC.SuccConnEstab.OrgHPSi + VS.RRC.SuccConnEstab.OgLPSig +
VS.RRC.SuccConnEstab.TmHPSi + VS.RRC.SuccConnEstab.TmLPSi + VS.RRC.SuccConnEstab.Rest) /
∑(VS.RRC.AttConnEstab.OrgConv + VS.RRC.AttConnEstab.OrgStr + VS.RRC.AttConnEstab.OrgInt +
VS.RRC.AttConnEstab.OrgBkg + VS.RRC.AttConnEstab.OrgSubs + VS.RRC.AttConnEstab.TrmConv +
VS.RRC.AttConnEstab.TrmStr + VS.RRC.AttConnEstab.TrmInt + VS.RRC.AttConnEstab.TrmBkg +
VS.RRC.AttConnEstab.OrgHPSig + VS.RRC.AttConnEstab.OrgLPSig + VS.RRC.AttConnEstab.TmHPSig +
VS.RRC.AttConnEstab.TmLPSig + VS.RRC.AttConnEstab.Rest)
•
Main Influencing Factors
Page 26

RRC Reject（ Power Congestion、CE Congestion、NodeB RL Fail、ALCAP AAL2 Fail、FP Fail、Code Congestion、Other）

RRC setup no response

Exclude the success rate of the WCDMA networks in covering the edge cell of the network.

The failed RRC connection setup due to the network congestion is excluded.

Exclude the RRC connection setup failure caused by network congestion
•
RAB Connection Setup
Success Rate
Test Method: stat.
 Stat. Number of RAB Setup Request in Point A of Figure1
 Stat. Number of RAB Setup Success in Point B of Figure1
 Stat. Number of RAB Setup Failure in Point B of Figure2
UE
RNC
CN
UE
RNC
RAB ASSIGNMENT
REQUEST
RAB ASSIGNMENT
REQUEST
A
A
RADIO BEARER
SETUP
RADIO BEARER
SETUP
RADIO BEARER
SETUP COMPLETE
RADIO BEARER
SETUP FAILURE
RAB ASSIGNMENT
RESPONSE
B
RAB ASSIGNMENT
RESPONSE
F1: Successful RAB setup procedure
CN
B
F2: RAB setup failure procedure
Page 27
CS RAB Connection Setup Success Rate
RAB connection setup success rate = Number of received CS RB setup
complete messages reported by the UE / Number of CS RAB assignment
requests sent by the CN x 100%
Note: The CS RAB connection setup success rate consists of the Voice
RAB connection setup success rate and the VP RAB connection setup
 Reference value of class A network
success rate.
Voice RAB connection setup success rate >= 99.5%, VP RAB
connection setup success rate >= 99.5%
 Formula for calculating the CS RAB connection setup success rate
1. Voice RAB connection setup success rate = 100 x
<VS.RAB.SucEstCSConv0.32.RNC> /
<VS.RAB.AttEstCSConv0.32.RNC>
2. VP RAB connection setup success rate = 100 x
<VS.RAB.SucEstCSConv32.64.RNC> /
<VS.RAB.AttEstCSConv32.64.RNC>
 In the Nastar software
Counter of the Voice RAB connection setup success rate:
RAB.SuccEstabCSConv0.32.RNC.Rate
Counter of the VP RAB connection setup success rate:
RAB.SuccEstabCSConv32.64.RNC.Rate
Page 28
PS RAB Connection Setup Success Rate
PS RAB Connection Setup Success Rate
PS RAB success rate = Number of received RAB setup complete messages of the
PS domain reported by the UE / Number of RAB assignment requests of the PS
domain sent by the CN x 100%
 Reference value of class A network
PS RAB connection setup success rate >= 98.5%
 Formula for calculating the PS RAB connection setup success rate
PS RAB connection setup success rate = 100 *
(RAB.SuccEstabPSNoQueuing.Conv +
RAB.SuccEstabPSNoQueuing.Strm +
RAB.SuccEstabPSNoQueuing.Intact +
RAB.SuccEstabPSNoQueuing.Bgrd +
RAB.SuccEstabPSQueuing.Conv + RAB.SuccEstabPSQueuing.Strm +
RAB.SuccEstabPSQueuing.Intact + RAB.SuccEstabPSQueuing.Bgrd) /
(RAB.AttEstabPS.Conv + RAB.AttEstabPS.Strm +
RAB.AttEstabPS.Intact + RAB.AttEstabPS.Bgrd)
 In the Nastar software
Counter of the PS RAB connection setup success rate:
Page 29
Call Completion Rate of the Calling Party
Call completion rate of the calling party (DT test)
Call completion rate of the calling party = Number of call completions of the calling party/Number of call requests of the calling party x 100%
Note:
The number of call requests of the calling party is the number of RRC Connection Requests initiated by the calling UE. If the RRC Connection Request is sent
for multiple times during a call, it is recorded once only.
The number of call completions of the calling party is the number of Alerting messages received by the calling UE.
Reference value of class A network
Voice >= 99%, VP >= 98%
Service
Voice/VP
Test conditions
①
②
③
Carry out the test in the area of Ec/Io>-11dB and RSCP>-90dBm.
Carry out the test in the case of low traffic, that is, the uplink load and the downlink load cannot exceed the planned load.
Exclude the access failure caused by the admission failure.
Exclude the access failure caused by problems not relating to access networks or by the UE.
Test methods
①
②
③
④
⑤
⑥
⑦
Collect statistics on the Voice or the VP separately.
Adopt WCDMA ONLY of SCANNER + Voice to consecutively make a call (short-time call).
The called party uses a fixed-line phone or the UE with strong signals.
The calling party makes a call. Then, the number of calling party requests is increased by 1. If the calling party hears the
ring back tone, the number of successful call setups is increased by 1. Otherwise, the call fails to be set up.
After the call is set up, maintain the conversation for 15 seconds. Then, the calling party hangs up. After 10 seconds, the
calling party makes a call again.
In the case of a call failure, makes a call after 30 seconds.
At least carry out the test 200 times.
Page 30
Call Completion Rate of the
Called
Party
Callcompletion
completion rate
of of
the the
called
party
= Number
call completions of the called
Call
rate
called
party
(DToftest)
party/ Number of call requests of the called party x 100%
Note: The number of call requests of the called party is the number of CM Service
Requests initiated by the calling UE.
The number of call connections of the called party is the number of Alerting messages
received by the calling UE.
Reference value of class A network
Voice >= 98%, VP >= 98%
Service
Voice/VP
Test conditions
①
②
③
④
⑤
⑥
Carry out the test in the area of Ec/Io>-11dB and RSCP>-90dBm.
Carry out the test in the case of low traffic, that is, the uplink load and the downlink load cannot exceed the specified load.
Exclude the access failure caused by the admission failure.
Exclude the access failure caused by problems not related to the access network or by the UE.
Exclude the paging failure caused by power-off of the UE.
Exclude the paging failure for the reason that the called party is busy.
Exclude the paging failure for the reason that the called UE is out of service.
Test methods
①
②
③
Collect statistics on the voice call and VP call separately. Adopt WCDMA ONLY of SCANNER + Voice/VP to consecutively
make a call (short-time call).
The calling party uses a fixed-line phone or the UE with strong signals. The calling party cannot uses a fixed-line phone during a
VP call.
31called party hears the
The calling party makes a call. Then, the number of call requests of the called party increases byPage
1. If the
ring back tone, the number of call completions of the called party increases by 1. Otherwise, the call fails to be set up.
PDP Activation Success Rate
PDP activation success rate (DT test)
PDP activation success rate = Number of successful PDP activations / Number of PDP activation requests x 100%
Note: There are two ways to define the number of PDP activation requests.
a)
Number of Activate PDP Context Requests initiated by the UE.
b)
Number of RRC Connection Requests initiated by the UE. The cause value contained in the request is the interactive service and background
service of the calling party and called party. If the RRC Connection Request is sent for multiple times during a call, it is recorded once only.
The number of successful PDP activations is the number of Activate PDP Context Accept messages received by the UE.
Reference value of class A network
>= 99％
Service
For the PDP activation success rate of R99, the PS service is established on the DCH. For the HSDPA, the PS service is established on the HSDPA.
Test conditions
①
②
③
④
Carry out the test in the area of Ec/Io > -11dB and RSCP > -90dBm.
Carry out the test in the case of low traffic, that is, the uplink load and the downlink load cannot exceed the specified load. Exclude the
access failure caused by the admission failure.
Exclude the access failure caused by problems not related to the access networks or by the UE.
The activation success rate of the service varies slightly with the rate. The service with a higher rate has a relatively lower activation rate.
Test methods
①
②
③
④
⑤
⑥
If the PS service is established on the DCH, adopt WCDMA ONLY of SCANNER + PS128K to consecutively perform the PDP activation
test, that is, make short-time calls. If the PS service is established on the HSDPA, adopt WCDMA ONLY of Scanner + HSDPA to
consecutively perform the PDP activation test.
Carry out the test or collect statistics based on the typical subscription rate.
Carry out the PDP activation test.
If the PDP activation is successful, deactivate the PDP after 10 seconds.
If the PDP activation fails, activate the PDP again after 10 seconds.
Calculate the number of successful PDP activations and the number of PDP activation requests.
Page 32
RAN KPI
Counters
Coverage
Accessibility
Retainability
Delay
Mobility
Page 33
Analysis of Call
Drop Rate
• Call drop rate is a KPI that telecom operators are required to be tested,
regardless of the WCDMA network or the GSM network. It is an important
counter for the analysis of the traffic measurement and DT/CQT.
• On the basis of the GSM network, the video call service is added to the
WCDMA network. Compared with the analysis of the call drop rate in the GSM
network, the analysis of the call drop rate in the WCDMA network is very
difficult. Therefore, you should make more efforts to analyze it during the
network optimization.
Page 34
CS Call Drop Rate (DT Test)
CS call drop rate (DT test)
CS call drop rate = Number of CS call drops / Number of CS call connections x 100%
Note: The number of CS call connections is the number of Alerting messages received by the UE after the service request is initiated
Number of CS call drops: From the viewpoint of the Um interface signaling recorded on the UE side, the number of CS call drops is
increased by 1 when one of the following messages of the Um interface is received during the conversation (in the call connection state).
(1) Any BCH message (system message)
(2) RRC Release message in which the release cause value is Not Normal
(3) CC Disconnect message, CC Release Complete message, or CC Release message in which the release cause value is Not Normal Clearing,
Not Normal, or Unspecified.
Reference value of class A network
Voice <= 1%, VP <= 1.5%
Test conditions
①
②
③
Carry out the test in the area of Ec/Io>-11dB and RSCP>-90dBm.
Carry out the test in the case of low traffic, that is, the uplink load and the downlink load cannot exceed the specified load.
Exclude call drops caused by network congestion.
Exclude call drops caused by problems not related to the access networks or by the UE.
Test methods
①
②
③
④
⑤
⑥
Adopt WCDMA ONLY of SCANNER + Voice/VP to consecutively make a call (long-time call).
During a voice/VP call, the called party is in the strong-signal area of RSCP > -80dBm and Ec/Io > -8dB
Set the call duration of the UE to 90 seconds and the waiting time after the call is terminated to 15 seconds.
If the call drop occurs, initiate a call after 30 seconds.
Calculate the number of call connections and the number of call drops.
At least, carry out the test 200 times.
Page 35
PS Call Drop Rate (DT Test)
PS call drop rate (DT test)
PS call drop rate = Number of PS call drops / Number of successful PDP activations x 100%
Note: The number of successful PDP activations is the number of Activate PDP Context Accept messages received by the UE
Number of PS call drops: From the viewpoint of the RRC signaling recorded on the UE side, the number of PS call drops is increased by 1 when
the air interface messages meet one of the following conditions in the data service connection state.
(1) The RRC Release message is not received, but the UE changes from the connection state (CELL_DCH, CELL_FACH, CELL_PCH,
URA_PCH) to the idle state (IDLE).
(2) The RRC Release message is received and the release cause value is not Normal or User Inactivity.
Reference value of class A network
<= 2%
Service
For the PS call drop rate of R99, the service adopts the planned PS full-coverage rate; for the PS call drop rate of HSDPA, the service is carried on HSDPA
and adopts the planned HSDPA cell margin rate.
Test conditions
①
②
③
Carry out the test in the area of Ec/Io>-11dB and RSCP>-90dBm.
Carry out the test in the case of low traffic, that is, the uplink load and the downlink load cannot exceed the specified load. Exclude call drops
caused by network congestion.
Exclude call drops caused by problems not relating to access networks or by the UE.
Test methods
①
②
③
④
⑤
For the PS call drop rate of R99, adopt WCDMA ONLY period of SCANNER + PS full-coverage rate to download the test (short-time call).
For HSDPA PS call drop rate, adopt WCDMA ONLY period of SCANNER + HSDPA to download the test (short-time call).
Carry out the test or collect statistics according to the full-coverage rate.
Perform the PDP activation. If the PDP activation is successful, the number of successful PDP activations is increased by 1.
Start the FTP server to download. After 90s, perform the PDP deactivation. If the call drop occurs during the download, the number of PS call
drops is increased by 1.
At least, carry out the test 200 times.
Page 36
Call Drop Rate (Stat.)
•
Test Method: stat.
 Stat. Number of Call Drop in Point A of Figure 1 and in Point A of Figure 2
 Definition of Call Drop: Abnormal RAB release triggered by RNC
RNC
CN
RNC
CN
IU RELEASE REQUEST
IU RELEASE COMMAND
RAB RELEASE REQUEST
IU RELEASE COMPLETE
A
A
F1: RAB release request procedure
F2: Iu connection release request procedure
Page 37
CS Call Drop Rate (Stat.)
CS call drop rate (stat.)
CS call drop rate = Number of abnormal CS releases/(Number of abnormal CS releases +
Number of normal CS releases)
 Reference value of class A network: Voice call
drop rate <=1%; VP call drop rate <=1.5%
 Formula for calculating the CS call droop rate:
CS AMR call drop rate ＝100*
(VS.RAB.Loss.CS.AMR) /
(VS.RAB.Loss.CS.AMR +
VS.RAB.Loss.CS.Norm.AMR)
VP call drop rate = 100 *
(VS.RAB.Loss.CS.Conv64K.RNC) /
(VS.Norm.Rel.CS.Conv.RB.64 +
VS.RAB.Loss.CS.Conv64K)
 In the Nastar software:
Counter of the CS AMR call drop rate:
VS.AMR.Call.Drop.Rate.RNC
Page 38
PS Call Drop Rate (Stat.)
PS call drop rate (stat.)
PS call drop rate = Number of abnormal PS releases initiated by the RNC/(Number of abnormal PS releases
initiated by the RNC + Number of normal PS releases initiated by the RNC)
 Reference value of class A network: PS call
drop rate <=3%
 Formula for calculating the PS call drop rate:
PS call drop rate =100 * ( <VS.RAB.Loss.PS.RF.RNC> +
<VS.RAB.Loss.PS.Abnorm.RNC> )/(VS.RAB.Loss.PS.RF +
VS.RAB.Loss.PS.Abnorm + VS.RAB.Loss.PS.Norm)
 In the Nastar software:
Counter of the PS call drop rate:
VS.PS.Call.Drop.Rate.RNC
 Influencing factors and deviation suggestions
1. Exclude the case that the equipment is
faulty.
2. Exclude the case of overload.
Page 39
•
•
•
•
Page 40
KPI Counters Related to Call
Drop Rate (Stat.)
Overall counter
 CS call drop
 CS call drop rate
 PS call drop
 PS call drop rate
Traffic class based counter
 Conversational
 Streaming
 Interactive
 Background
Typical bearer based counter
 AMR speech
 CS 64 kbit/s
 PS 64 kbit/s
 PS 128 kbit/s
 PS 384 kbit/s
Main Influencing Factors
 RF loss
 RLC reset
 AAL2 failure
 Other cause
RAN KPI
Counters
Coverage
Accessibility
Retainability
Delay
Mobility
Page 41
Delay
• Delay is mainly tested by the call quality test (CQT). There is no
great difference in the GSM network and the WCDMA network.
For the CS services, the VP service is added to the WCDMA
network. For the PS services, more channel occupation modes
and multiple rates are applied to the WCDMA network.
• Delay is applicable to the delivery or when customers perform
the network CQT survey. Daily KPI monitor and network
optimization are mainly performed on the basis of the
exported data of the KPIs.
Page 42
•
AMR/VP Voice
Delay
AMR/VP delay test method: CQT

•
Test conditions



•
Connection delay: Refers to the interval from the time when the UE sends the RRC
Connection Request for the first time to the time when the UE receives an Alerting
message.
1. UEs of the calling party and called party are in the strong-signal area of RSCP > 90dBm and Ec/Io > -11dB.
2. Both the calling party and called party are in the local office.
3. When the static test is performed, the location of UE remains unchanged.
Test methods
1. If you use a stopwatch to calculate the delay, the timing begins when you press the dial
key and ends when a ring back tone is heard.
2. If the ring back tone from the peer end is not received, the call is defined to be
unsuccessful and this delay is not recorded.
3. Record call-related messages (the interval from the time when the UE sends the RRC
Connection Request to the time when the UE receives an Alerting message).
4. At least, carry out the test 20 times. Then, calculates the average delay.
5. Change a cell to perform the test at five locations.
Page 43
•
VP Image
Delay
CQT VP image delay test method: CQT

•
Connection delay: Refers to the interval from the time when the UE sends the RRC
Connection Request for the first time to the time when the first frame image appears.
Test conditions
1. UEs of the calling party and called party are in the strong-signal area of RSCP > -90dBm
and Ec/Io > -11dB.
2. Both the calling party and called party are in the local office.
3. When the static test is performed, the location of UE remains unchanged.
•
Test methods
1. Record the time when you see the first frame image after you press the talk key.
2. If the Alerting message from the peer end is not received, the call is defined to be
unsuccessful and this delay is not recorded.
3. At least, carry out the test 20 times. Then, calculates the average delay.
4. Change a cell to perform the test at five locations.
5. Exclude the excessive access delay caused by non-network reasons.
Page 44
AMR/VP Call
Delay
UE->UE
Means：4.5s
95％<5.5s
UE->PSTN
Means：3.3s
95％<3.54s
UE->UE
Means：4.5s
95％<5.5s
Voice
VP-Alerting
VP-display
UE->UE
Means：9s
(The VP delay difference is
great between different UEs)
Page 45
Ping Delay
•
Ping delay test method: CQT

•
•
Ping delay: Refers to the time when the response message is received when the Ping request is
sent.
Test conditions
–
RSCP>-90dBm， Ec/Io>-11dB ； When the static test is performed, the UE should be placed in the
area with good coverage, and then is connected to the test equipment. RSCP>-90dBm, Ec/Io>11dB
–
When the uplink or downlink TCH is mapped onto the DCH, the DCCC algorithm must be disabled
to perform the Ping delay test. If the DCCC algorithm is enabled, the speed may decrease during
the Ping test. In this case, the delay may increase.
–
The server adopts the Windows operating system, and the Ping server is directly connected to the
Gi interface.
Test methods
1. According to the requirements, the size of the Ping packets should be 0 bytes and 1460 bytes.
2. Exclude the excessive delay caused by the non-network reason.
3. If the timeout occurs, the average value of the delay should be calculated manually.
Do not use the average value of the delay displayed in the CMD command.
4. At least, carry out 100 times. The average value is used as the value of the delay.
5. Change a cell to perform the test at five locations.
6. Exclude the excessive access delay caused by non-network reasons.
Page 46
Reference KPIs Related
to Ping Delay
Counter Name
Definition
Service
Reference Value of
Delay
DCH Ping
It specifies the ping delay when the uplink or
downlink TCH is mapped onto the DCH.
UL 128/DL 384
0 bytes: 170 ms
1460 bytes: 380 ms
FACH Ping
It specifies the ping delay in the FACH state.
RACH/FACH
0 bytes: 200 ms
HSDPA Ping
It specifies the ping delay in the HSDPA
channel.
UL 128k/DL HSDPA
0 bytes: 90ms
1460 bytes: 150ms
FACH to DCH Ping
It specifies the Ping delay when the state of
the TCH transits from the FACH state to the
DCH state after the Ping is performed.
Ping 1460 Bytes
1300 ms
FACH to HSDPA
Ping
It specifies the Ping delay when the state of
the TCH transits from the FACH state to the
HSDPA state after the Ping is performed.
Ping 1460Bytes
1000 ms
Idle to DCH Ping
It specifies the Ping delay when the state of
the TCH transits from the idle state to the
DCH state after the Ping is performed.
Ping 1460 Bytes
2800ms
Idle to HSDPA Ping
It specifies the Ping delay when the state of
the TCH transits from the idle state to the
HSDPA state after the Ping is performed.
Ping 1460 Bytes
2650 ms
Page 47
RAN KPI
Counters
Coverage
Accessibility
Retainability
Delay
Mobility
Page 48
Analysis of Handover
Success Rate
•
Like the call drop rate, the data of the handover success rate needs to be also exported from
the test software and the drive test (DT). Call drop rate and handover success rate are
important KPIs for engineering delivery.
•
According to the attributes of the target cell, the KPIs of the WCDMA network are
categorized into the following: soft handover success rate, intra-frequency/inter-frequency
hard handover, inter-RAT handover. During the analysis process, the soft handover success
rate (similar to the G-BSC internal handover) is mainly analyzed. The intra-frequency
handover occurs in the edge of the Radio Access Network (RAN) (refer to the GSM outgoing
BSC handover). The inter-frequency handover occurs in the multi-band network (refer to the
GSM dual-band handover). The inter-RAT handover indicates the WCDMA-to-GSM handover.
•
Compared with the GSM network, the handover counters related to the PS services is added
to the WCDMA network. The procedure for the PS service handover in the WCDMA network
can refer to the procedure for the CS handover in the GSM network.
Page 49
Soft Handover Success Rate
Soft handover success rate = Number of successful soft handovers / Number of
Soft handover success rate (DT & Stat.)
soft handover attempts x 100%
Note: The number of soft handover attempts is the number of Active Set Update
messages sent by the RNC
The number of successful soft handovers is the number of Active Set Update
Reference value of class A network
Complete messages received by the RNC.
>= 99.5%
Service
CS & PS
Test conditions
①
②
③
④
Perform the test in the area of Ec/Io>-11 and RSCP>-90dBm.
Carry out the test in the case of low traffic, that is, the uplink load and the downlink load do exceed the specified load.
Exclude the failure to add handover links caused by network congestion.
Exclude the handover failure caused by the failure to establish the user plane.
Exclude the handover failure caused by the UE.
Test methods
①
②
③
④
⑤
Perform a long-conversation test by adopting different services, and place the UE of the called party in the strong-signal
area.
The RNC starts the single user signaling tracing of the tested UE.
The UE records the RRC signaling process.
Count the number of activity sets to be updated sent by the RNC and the number of updated activity sets received by the
RNC.
If the conversation is interrupted during the test, re-establish the conversation.
Counters for Traffic Measurement
①
②
Soft handover counter: VS.SHO.Succ/VS.SHO.Att.RNC
Counter of the soft handover success rate in the Nastar software: VS.SHO.Succ.Rate
Page 50
Soft Handover
Success Rate
•
Measurement Point of Soft Handover
 Figure 1: procedure for successful soft handover
 Figure 2: procedure for failed soft handover
 Figure 3: procedure for soft handover timeout
Main
Influencing Factors
Handover
Delay
parameter
factor
Movement
speed
Interference
Signal
UE
RNC
MEASUREMENT REPORT
UE
RNC
change
UE
RNC
MEASUREMENT REPORT
MEASUREMENT REPORT
A
ACTIVE SET UPDATE
A
A
ACTIVE SET UPDATE
ACTIVE SET UPDATE
B
ACTIVE SET UPDATE COMPLETE
B
B
WAITING ACTIVE SET UPDATE
COMPLETE TIME OUT
ACTIVE SET UPDATE FAILURE
C
factor
C
C
F1: Successful soft handover procedure
F3: Soft handover timeout p
F2: Failed soft handover procedure
Page 51
Inter-Frequency Hard Handover Success Rate
Inter-frequency hard handover success rate (DT & stat.)
Inter-frequency hard handover success rate = Number of successful inter-frequency hard handovers / Number of inter-frequency hard handover attempts x 100%
Note: The number of successful inter-frequency hard handovers is the number of physical channels (used for inter-frequency hard handover)
reconfiguration completion messages received by the RNC.
The number of inter-frequency hard handover attempts is the number of physical channel (used for inter-frequency hard handover) reconfiguration
requests sent by the RNC
Reference value of class A network
Voice >= 95%, VP >= 93%, PS64K > 93%, PS144K >= 90%, PS384K >= 88%
Service
CS & PS
Test conditions
①
②
③
④
Perform the test in the area of Ec/Io > -11dB and RSCP > -90dBm.
Carry out the test in the case of low traffic, that is, the uplink load and the downlink load cannot exceed the specified load. Exclude the
failure to add handover links caused by network congestion.
Exclude the hard handover failure caused by the failure to establish the user plane.
Exclude the hard handover failure caused by the UE.
Test methods
①
②
③
④
⑤
Make a long-time CS call or download the PS service.
The RNC starts the single-UE signaling tracing of the UE. The UE records the RRC signaling process.
Calculate the number of physical channels (used for inter-frequency hard handover) reconfiguration requests sent by the RNC and the
number of physical channel (used for inter-frequency hard handover) reconfiguration completion messages received by the RNC .
If the conversation is interrupted during the test, re-establish the conversation.
At least, perform the test 20 times in each handover scenario.
Page 52
Inter-Frequency Handover
Success Rate (Outgoing)
• Measurement Point of Hard Handover
UE
Figure 1: procedure for successful
hard handover
Figure 2: procedure for failed hard
handover
Figure 3: procedure for hard
handover timeout
NodeB
RNC
MEASUREMENT REPORT
A
RADIO LINK SETUP
REQUEST
RADIO LINK SETUP
RESPONSE
PHYSICAL CHANNEL RECONFIGURATION
PHYSICAL CHANNEL
RECONFIGURATION COMPLETE
B
C
F1: Successful hard handover
procedure
UE
RNC
UE
RNC
PHYSICAL CHANNEL
RECONFIGURATION
PHYSICAL CHANNEL
RECONFIGURATION
PHYSICAL CHANNEL
RECONFIGURATION FAILURE
WAITING PHYSICAL CHANNEL
RECONFIGURATION TIME OUT
A
A
F2: Failed hard handover
procedure
F3: Hard handover timeout
procedure
Page 53
AMR Inter-RAT Handover Success
Rate
(DT
&
stat.)
AMR inter-RAT handover success rate (DT & stat.)
Collect statistics on the UE side
AMR inter-RAT handover success rate (outgoing) = Number of successful inter-RAT handovers / Number of inter-RAT handover attempts x 100%
Note: The number of successful inter-RAT handovers is the number of Handover Complete messages sent by the UE.
The number of inter-RAT handover attempts is the number of Handover from UTRAN Command messages received by the UE.
In the flow chart of inter-RAT handover of the CS:
 A and B indicate the handover preparation process.
 C and D indicate the handover execution process.
UE
SRNC
CN
RELOCATION
REQUIRED
A
RELOCATION
COMMAND
HANDOVER FROM
UTRAN COMMAND
B
C
IU RELEASE
COMMAND
D
Successful inter-RAT HO outgoing procedure
Page 54
AMR Inter-RAT Handover Rate (Outgoing)
AMR inter-RAT handover rate (outgoing)
Reference value of class A network
Voice >= 97%
Test conditions
①
②
③
④
⑤
⑥
⑦
The speed of the WCDMA network signal decreases by less than 30dB within 3 seconds.
Exclude the handover scenario in the elevator.
When the handover occurs, RxLex of the GSM network signal must be greater than -85dBm.
Exclude handover failures caused by GSM network congestion.
Exclude handover failures caused by GSM networks.
Exclude handover failures caused by the UE.
It is not a failure when the UE returns to the WCDMA network during the handover.
Test methods
①
②
③
④
⑤
Determine three to five scenarios in which the UE is handed over from the WCDMA network to the GSM network. For example, the GSM
network is equipped with the indoor distribution system, and the WCDMA network does not have the indoor distribution system.
Make a long-time Voice call for a test and move the UE from the WCDMA coverage area to the GSM coverage area.
Check whether the UE is handed over from the WCDMA network to the GSM network, that is, check whether the icon on the screen changes
from WCDMA to GSM. If the conversation is not interrupted, the number of successful handovers is increased by 1. If the conversation is
interrupted, the number of failed handovers is increased by 1. Return to the WCDMA area. Perform the cell reselection forcibly for the UE to
camp on a WCDMA cell. Record signaling messages on the UE side and the RNC side.
At least, perform the test over 20 times in each handover scenario.
Formula for calculating the handover success rate: Number of successful handovers/(Number of successful handovers + Number of failed
handovers)
Counters for traffic measurement and calculation formula
①
②
Success rate of inter-RAT handovers (outgoing) in the CS domain = 100 x (VS.IRATHO.SuccCSOut) / (VS.IRATHO.AttCSOut.RNC)
In the Nastar software, the counter of the success rate of inter-RAT handovers (outgoing) in the CS domain is
VS.IRATHO.SuccCSOut.RNC.Rate.
Page 55
•
PS Inter-RAT Handover Success
Rate (Outgoing)
PS inter-RAT handover success rate (outgoing) DT
 Formula for calculating PS inter-RAT handover success rate (outgoing)
PS inter-RAT handover success rate (outgoing) = Number of successful PS inter-RAT handovers / Number of PS inter-RAT handover attempts x 100%
Note: The number of successful PS inter-RAT handovers is the number of Routing Area Update Complete messages sent by the UE.
The number of PS inter-RAT handover attempts is the number of CELL CHANGE ORDER FROM UTRAN messages received by the UE.
In the flow chart of the PS inter-RAT handover,

A and B indicate the handover request process.

C and D indicate the successful handover process.
UE
SRNC
CN
MEASUREMENT
REPORT
CELL CHANGE
ORDER FROM
UTRAN
A
B
IU RELEASE
COMMAND
C
IU RELEASE
COMPLETE
D
UTRAN-controlled cell change outgoing procedure
Page 56
PS Inter-RAT Handover Success Rate (Outgoing)
 PS inter-RAT handover success rate (outgoing) DT
Reference value of class A network
>= 96%
Test conditions
① Exclude the handover scenario in the elevator.
② The WCDMA network signal speed must decrease by less than 30dB within
3 seconds.
③ When the handover occurs, RxLex of the GSM signal must be greater than 85dBm.
④ Exclude handover failures caused by GSM network congestion.
⑤ Exclude handover failures caused by GSM networks.
⑥ Exclude handover failures caused by the UE.
⑦ It is not a failure when the UE returns to the WCDMA network during the
handover.
Test methods
① Place the UE in the area where the inter-RAT handover can occur. The UE
camps on the WCDMA cell.
Page 57
PS Inter-RAT Handover Success Rate (Stat.)
inter-RAThandover
handover success
Number of successful PS inter-RAT handovers /
PSPSinter-RAT
successrate
rate= (stat.)
Number of PS inter-RAT handover attempts
Note: The number of PS inter-RAT handover attempts is the number of CELL
CHANGE ORDER FROM UTRAN messages sent from the RNC to the UE.
The number of successful PS inter-RAT handovers is the number of IU
RELEASE COMMAND messages received by the RNC in which the cause value
is Successful
or Normal Release.
 Reference
value ofRelocation
class A network
PS inter-RAT handover success rate:
>= 90%
 Formula for calculating the PS inter-RAT
handover success rate
PS inter-RAT handover success rate =
100 x
(VS.IRATHO.SuccPSOutUTRAN.RNC)
/ (VS.IRATHO.AttPSOutUTRAN.RNC)
 In the Nastar software
Counter of the PS inter-RAT handover
Page 58
Conclusion
•
•
•
•
•
Page 59
Understand the test purpose of the KPI
Know the application scenario of the KPI
Master the test method of the KPI
Compared with G-KPI, understand the
definition of the KPI
Understand the KPI baseline
Thank You
www.huawei.com