Huawei to Nokia RAN SWAP KPI
Mapping Information Package
Egidijus Janusevicius
egidijus.janusevicius@nokia.com
2021.01.29 (Prepared in Nov’20)
STRICTLY FOR NOKIA INTERNAL USE ONLY
Version 3.0
1
© 2020 Nokia
Nokia internal use
Huawei to Nokia RAN KPIs information package
Contents
1. Nokia RFQ KPI Commitment Targets: 2G/3G/4G/5G -> LINK
2. Main KPI mapping Info-Set per technology & service:
•
2G (Voice & Data) -> LINK
•
3G (Voice & Data) -> LINK
•
4G (VoLTE & Data) -> LINK
•
5G (Data/NSA) (unlikely swap / likely modernization) -> LINK
3.
4.
5.
6.
Experience based KPI Mapping process notes
Mapping Examples
Various Nokia Material References
Huawei Reference Documentation -> LINK
2
© 2020 Nokia
Nokia internal use
1. Nokia KPI Commitment Targets
MN Guidelines
Nokia RFQ KPI Commitment Guideline please refer to -> LINK
KPI Commitment Targets per Technology: subject to approval as per Guideline rules
•
•
•
•
2G -> LINK
3G -> LINK
4G -> LINK (FDD), LINK (TDD)
5G -> LINK
Guideline & Target documents cover:
•
•
Global “out of the box” KPI values
Global “plus optimization” values (considering NCS services)
Note:
KPI target values are applicable for Nokia Standard KPI definitions. Any non-Nokia or custom KPIs
values cannot be compared w/o diligent mapping process taken.
NEI Performance Benchmarker -> LINK (Highly Recommended Source for KPI Level Assessment)
3
© 2020 Nokia
Nokia internal use
2.1. 2G Main KPI Mapping Available Data Summary
Identified available mapped metrics
Click on the relevant KPI link (>> or LINK) to access the reference source
TECHNOLOGY
2G KPI MAPPING REPOSITORY
SERVICE
GENERAL
VOICE
DATA
DOMAINAVAILABILITY
MOBILITY
OTHER
TRAFFIC
ACCESSIBILITY RETAINABILITY TRAFFIC
ACCESSIBILITY RETAINABILITY THROUGHPUT
# / NW
LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT
4
MPD &
MCDR
2
SYV FI
>>
n/a
>>
n/a
3
WINGZ
BE
n/a
n/a
n/a
>> Traffic
>>
Under
>> Monitoring
KPIs
% from Total
NOTES
LINK TO DOCUM
DOCU ENT
MENT TYPE
>>
>>
n/a
>>
n/a
n/a
2(3) slides per KPI:
H.vs.N.
LINK
PPT
>>
>>
n/a
n/a
n/a
n/a
KPI Mapping sheet:
line 2-4
LINK
XLS
>> Line 4-5
>> Line 8
Under
>> Monitoring
KPIs
>> Line 6
>> Line 9-10
n/a
LINK
XLS
4
KARTEL
KZ
>> Line 19
Under
>> Monitoring
KPIs
5
PLAY PL
>> Line 3
>> Line 20-23
>> Line -24-31 >> Line 4-5
>> Line 13-14
>> Line 17
>> Line 6-7
>> Line 15-16
>> Line 18-19
>> Line 8-11
See 2G sheet
LINK
XLS
7
VEON KZ n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
Only Nokia values
for reference
LINK
PPT
8
BT UK
>> Line 27-28
>> Line 34-41
>> Line 32
>> Line 16-24
>> Line 26
n/a
>> Line 3
>> Line 25
n/a
See 2G sheet
LINK
XLS
n/a
NO KPI MAPPING.
EVENT COUNTER
MAPPING ONLY
LINK
11
NONAME
…
Y
…
HEDEX
>> Line 15
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
LINK to Default KPIs & Counter Folder
© 2020 Nokia
Nokia internal use
n/a
2.2. 3G Main KPI Mapping Available Data Summary
Identified available mapped metrics
Click on the relevant KPI link (>> or LINK) to access the reference source
TECHNOLOGY
3G KPI MAPPING REPOSITORY
SERVICE
GENERAL
VOICE (CS)
DATA (PS)
ALL
NOTES
DOMAINAVAILABILITY MOBILITY
TRAFFIC
ACCESSIBILITY RETAINABILITY OTHER (CS)
TRAFFIC
ACCESSIBILITY RETAINABILITY THROUGHPUT OTHER (PS)
OTHER
NE-BORDER
# / NW
LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT
MCDR
only RAB
only RAB &
see notes
2(3) slides per KPI:
customized
>>
>>
>>
>> MPD
>>
>> KPI (!)
>>
>> (note: sl.
2
SYV FI
n/a
n/a
n/a
n/a
n/a
stage
STS (!)
sl.34
H.vs.N.
32)
3
4
5
WINGZ
BE
KARTEL
KZ
n/a
n/a
n/a
n/a
>> SHO/ IRAT >>
>>
Under
Monitoring
KPIs
>>
>>
separate
>>
RRC & RAB
5
PLAY PL
>>
>> SHO/IRAT
>>
only RAB
>>
stage
7
VEON
KZ
n/a
n/a
n/a
n/a
…
Y
…
HEDEX
n/a
n/a
>> Attempts
Under
separate
>> Monitoring >>
>>
RRC & RAB
KPIs
>>
n/a
n/a
n/a
n/a
n/a
>>
>>
only RAB
>>
stage
>>
n/a
n/a
n/a
incomplete
mapping
"PCH" KPI
incomplete
mapping
LINK to Default KPIs & Counter Folder
© 2020 Nokia
Nokia internal use
n/a
LINK
DOCUM
TO
ENT
DOCU
TYPE
MENT
LINK
PPT
n/a
n/a
n/a
KPI Mapping sheet:
line 9-11
LINK
XLS
>>
RTWP:
>> calculation n/a
?
PS DCR incomplete
KPI mapping
LINK
XLS
See 3G sheet
LINK
XLS
Only Nokia values
for reference
LINK
PPT
HSDPA
Attempts
>>
n/a
>> RRC stage
Various
>>
KPIs
n/a
n/a
n/a
n/a
2.3. 4G Main KPI Mapping Available Data Summary
Identified available mapped metrics
Click on the relevant KPI link (>> or LINK) to access the reference source
TECHNOLOGY
4G KPI MAPPING REPOSITORY
SERVICE
GENERAL
VOLTE
DATA
DOMAIN AVAILABILITY
MOBILITY
OTHER
TRAFFIC
ACCESSIBILITY RETAINABILITY OTHER
TRAFFIC
ACCESSIBILITY RETAINABILITY THROUGHPUT OTHER
# / NW
LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT
6
1
BELL CN
n/a
>> page 35-55 n/a
n/a
>> page 21-30
>> page 31-34 n/a
2
SYV FI
>>
n/a
n/a
n/a
>>
>>
>>
3
WINGZ
BE
n/a
n/a
n/a
n/a
n/a
n/a
n/a
4
KARTEL
KZ
>>
5
PLAY PL
>> line 3
7
Under
Monitoring
KPIs
MPD &
MCDR
NOTES
LINK TO DOCUM
DOCUM ENT
ENT
TYPE
n/a
>> page 2-6
>> page 7-20
n/a
n/a
Diligent mapping
LINK
PDF
n/a
>>
>>
>> ref LTE5702 n/a
2(3) slides per KPI:
H.vs.N.
LINK
PPT
n/a
>>
>>
n/a
KPI Mapping sheet:
lines 5-6
LINK
XLS
LINK
XLS
Under
Monitoring
KPIs
>> line 33-35
n/a
>> line 36
>>
Under
Monitoring
KPIs
n/a
>> line 37-39
n/a
n/a
n/a
n/a
n/a
>>
>> line 15-21
n/a
n/a
n/a
n/a
n/a
>> line 4-5
>> line 10-13
>> line 14
>> line 6-8
n/a
See 4G sheet
LINK
XLS
VEON KZ n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
Only Nokia values for
reference
LINK
PPT
8
BT UK
>> line 4
>> line 11-15
>> line 20-34
n/a
>> line 6
>> line 10
>> line 25,30
>> line 18-19
>> line 5
>> line 9
>>
>> line 20-34
See 4G sheet.
Diligent Mapping
LINK
XLS
9
MF RU
n/a
>>
>> RRC
n/a
n/a
n/a
n/a
n/a
>> ERAB
>>
n/a
LINK
PPT
11
NONAME
…
Y
…
HEDEX
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
LINK to Default KPIs & Counter Folder
© 2020 Nokia
Nokia internal use
n/a
n/a
n/a
line 17-18,
26-27
>> RRC DCR
n/a
NO KPI MAPPING. EVENT
COUNTER MAPPING
ONLY
LINK
2.4. 5G Main KPI Mapping Available Data Summary
Identified available mapped metrics
Click on the relevant KPI link (>> or LINK) to access the reference source
TECHNOLOGY
5G KPI MAPPING REPOSITORY
SERVICE
DATA
DOMAIN AVAILABILITY
TRAFFIC
ACCESSIBILITY RETAINABILITY MOBILITY
THROUGHPUT ENDC
OTHER
# / NW
LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT
7
NOTES
LINK TO DOCUM
DOCUM ENT
ENT
TYPE
1
BELL CN
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
Only Nokia values
for reference. Page
8-14
LINK
PDF
2
SYV FI
n/a
n/a
n/a
n/a
n/a
n/a
n/a
n/a
Only Nokia KPI
proposals. Slide 5659
LINK
PPT
8
BT UK
>> Line 3
n/a
>> Line 4-6
>> Line 10
>> Lines 7-9
>> Line 11
>>
LINK
XLS
…
Y
…
HEDEX
LINK to Default KPIs & Counter Folder
© 2020 Nokia
Nokia internal use
Column C:
filter L
>>
KPI Details 2 documents:
contractual &
sheet
extended KPIs
3. Important KPI mapping process notes
Experience from other swaps
•
Always know what can be achieved with Nokia KPI, however this does not equal to what HUA/E/ZTE/S achieves with THEIR or customer’s CUSTOM KPIs.
•
KPI mapping activity is essential & is a must before swap & during swap’s early stages – it is in the interest of both parties (Customer & Nokia) to make sure it happens so
performance can be monitored/assessed and addressed in a proper professional way.
•
Be aware that KPI changes might need to be proposed for the HUA/CUSTOMER KPIs due to the scenarios that Nokia might not having certain specific counters used in the
existing metric by customer or HUA, or HUA/CUSTOMER KPI does not fully cover metric in question (while Nokia does).(ref. BT UK LTE KPIs)
•
Always question mapped proposed KPIs, metrics, SQIs. Always. Do not rely just on the KPI name, “seem to be self explanatory counter name” & “for sure to be passed”
reference value – traffic layering changes might impact performance values for example .
•
Split KPIs into “components”: RRC, RAB/eRAB/NAS, STS & etc. Check weighting used in KPIs (per traffic volume, per event volume, else)
•
Split KPIs into single counters for the analysis (nom/denom, counters per detailed reason, check for sub-counter availability)
•
Consider that not all “scenarios” are pegged under the counter in question (such as RRC repetitions, RAB RELOCs, etc.)
•
Cross check information against counter description & signaling flow chart: exact counter pegging point & reason.
•
Be aware of possibly “cooked” KPIs: for example when 0.2% of traffic has 80% weight on the KPI (for example PS RAB & STS volume impacts).
•
Always read notes for the KPI/ counter in question in the reference documents provided with the slide set to be aware of possible risks & prepare contingency plan.
•
Assess current SW level against reference KPI SW levels (example ORF KPI document) – additional counters might be included or removed – applies to all sides.
•
Consider that “National” type KPI definition could be presented (for the reason of simplified multi vendor management) & it might be not even possible to map the KPI in
logic or full completeness.
•
Traffic Layering strategy changes will impact achievable KPI levels (donut effects, loss of volumes, & etc.) – negative impact on CSSR & DCRs for the layers loosing “good”
traffic & staying with “failure” scenarios
•
Limit number of KPIs contractual / add monitoring / introduce critical KPIs & thresholds.
•
If possible (make it possible): performance DT based benchmark before swap to be repeated after swap for more end user centric performance comparison. Diligent testing
scenario preparation is a must.
•
SQI/NQI topic shall be considered
8
© 2020 Nokia
Nokia internal use
4.1. 3G KPI example mapping (for reference only)
To consider
Very often in Huawei used KPIs
the are single counter for the
event like drops of failures.
In Nokia counters per drop or
failure reason are usually used.
It is understood that Huawei also
has detailed reason counters per
failure and drop.
Function Subset Name
Counter
RAB Release Measurement <per Cell>(RAB.RelAbnormal.Cell)VS.RAB.AbnormRel.CS
When running the KPI
mapping’s fine tuning – it would
be advisable to assess
performance on the sub-counter
level (& against the totals) to
understand the actual gaps if any.
9
RAB Release Measurement <per Cell>(RAB.RelAbnormal.Cell)VS.RAB.AbnormRel.CS.RF
RAB Release Measurement <per Cell>(RAB.RelAbnormal.Cell)VS.RAB.NormRel.CS
RAB Release Measurement <per Cell>(RAB.RelAbnormal.Cell)VS.RAB.AbnormRel.CS.IuTNL
Counter Name
Sub Counter
VS.RAB.AbnormRel.CS.RF
VS.RAB.AbnormRel.CS.OM
VS.RAB.AbnormRel.CS.Preempt
VS.RAB.AbnormRel.CS.UTRANgen
VS.RAB.AbnormRel.CS.OLC
VS.RAB.AbnormRel.CS.IuTNL
VS.RAB.AbnormRel.CS.IuupFail
VS.RAB.AbnormRel.CS.RF.ULSync
VS.RAB.AbnormRel.CS.RF.UuNoReply
VS.RAB.AbnormRel.CS.RF.SRBReset
VS.RAB.AbnormRel.CS.CN
VS.RAB.NormRel.CS.UEGen
VS.RAB.NormRel.CS.UEReEst
VS.RAB.NormRel.CS.NetOpt
VS.RAB.AbnormRel.CS.IuAAL2
Counter Description
VS.RAB.AbnormRel.CS.RF.NodeShare
VS.RAB.AbnormRel.CS.RF.IubFlex
VS.RAB.AbnormRel.CS.RF.RNC
Number of CS RABs Released Abnormally due to Radio Link Failure for UEs Whose Load Is Shared Between Physical RNC Nodes
Number of CS RABs Released Abnormally for Iub-Flex UEs due to Radio Link Failures
Number of CS RABs Abnormally Released due to RF cause for RNC
VS.RAB.AbnormRel.CS.RF.ULSync
VS.RAB.AbnormRel.CS.RF.UuNoReply
VS.RAB.AbnormRel.CS.RF
VS.RAB.AbnormRel.CS.RF.SRBReset
Number of CS RABs Abnormally Released Due to RF for Cell (Uplink Synchronization Fail)
Number of CS RABs Abnormally Released Due to RF for Cell (Failure in the Radio Interface Procedure)
Number of CS RABs Abnormally Released Due to RF for Cell
Number of CS RABs Abnormally Released for Cell (Signaling RLC Reset)
© 2020 Nokia
Nokia internal use
?
4.2. 3G PS CSSR% Example
NPO would call it “cooked”. Informal Note
Huawei: 3G PS CSSR (%) = {100}*[(VS.RAB.SuccEstabPS.Conv +VS.RAB.SuccEstabPS.Bkg + VS.RAB.SuccEstabPS.Int +
VS.RAB.SuccEstabPS.Str + VS.DCCC.P2F.Succ + VS.DCCC.URAP2F.Succ + VS.DCCC.P2D.Succ +
VS.DCCC.URAP2D.Succ)/(VS.RAB.AttEstabPS.Conv + VS.RAB.AttEstabPS.Bkg +VS.RAB.AttEstabPS.Int + VS.RAB.AttEstabPS.Str +
VS.DCCC.P2F.Att +VS.DCCC.URAP2F.Att + VS.DCCC.P2D.Att + VS.DCCC.URAP2D.Att)]
3G PS SSR% = {100}*(sum(rab_acc_comp_ps_strea + rab_acc_comp_ps_inter + rab_acc_comp_ps_backg +
RAB_ACC_COMP_PS_CONV + succ_pch_dch_trans_umrlc – cell_update_succ_cs_call + denom_st_trans_time_pch_fach)
/ (sum(rab_stp_att_ps_strea + rab_stp_att_ps_inter + rab_stp_att_ps_backg + RAB_STP_ATT_PS_CONV + att_pch_dch_trans_umrlc cell_update_att_cs_call + att_pch_to_fach)
Several issues with the PS CSSR KPI:
1.
KPI does not include RRC Stage – incomplete visibility of the service performance
2.
KPI does not include NAS stage (actually, same as standard Nokia KPI) – incomplete visibility of the service performance
3.
KPI is built by merging RAB & STS attempts: (into 1 “bucket”)
1.
2.
3.
4.
5.
In Nokia RAB is not released nor setup during STS (simplified statement)
In this KPI, depending on the T1 & T2 timers used, STS attempt weight against RAB attempts could be significant raging from 1:1 to 1:5
or 1:8. This potentially could hide the RAB performance issues.
STS counter pegging mapping cannot be completely mapped
STS counter pegging exact scenario might be very different: especially the SUCC part of it.
RAB RELOCATION scenario counting is not clear: Nokia has different counters for the RELOC (inter-RNC/PSIHO from 4G)
Similar RAB+STS logic has been used for the PS DCR% KPI.
10
© 2020 Nokia
Nokia internal use
4.3. 4G Data Accessibility%
Split Example
Huawei 4G CSSR% =
100*(L.RRC.ConnReq.Succ / L.RRC.ConnReq.Att)*
(L.S1Sig.ConnEst.Succ / L.S1Sig.ConnEst.Att) *
(L.E-RAB.SuccEst / L.E-RAB.AttEst)
RRC Stage
S1 Stage
E-RAB Stage
Nokia 4G CSSR% =
100*((SIGN_CONN_ESTAB_COMP / (SIGN_CONN_ESTAB_ATT_MO_S + SIGN_CONN_ESTAB_ATT_MT +
RRC Stage
SIGN_CONN_ESTAB_ATT_MO_D + SIGN_CONN_ESTAB_ATT_DEL_TOL + SIGN_CONN_ESTAB_ATT_HIPRIO +
SIGN_CONN_ESTAB_ATT_EMG)) *
S1 Stage
(S1_SIGN_CONN_ESTAB_SUCC_UE / S1_SIGN_CONN_ESTAB_ATT_UE) *
(EPS_BEARER_SETUP_COMPLETIONS / EPS_BEARER_SETUP_ATTEMPTS))
E-RAB Stage
Several comments on the PS CSSR KPI:
1.
KPI does include RRC, S1 & ERAB setup stages – seems to provide the extended visibility of PS performance
2.
In some NW’s customer uses only “RRC*S1” (BELL CAN), “RRC*ERAB” (SYV), in other NWs 2-3 KPI are used to for
performance monitoring purposes
3.
RRC counter pegging shall be x-checked against RRC repetition counting (ref. BELL Canada: Hua counts repetitions under
“regular” counter)
4.
ERAB stage shall be x-checked against Initial & Additional ERAB attempts counting (ref. SYC FIN: proposed only INI counter
usage)
11
© 2020 Nokia
Nokia internal use
4.4. 4G KPI example mapping
To consider
Very often in Huawei used KPIs the
are single counter for the event like
drops of failures.
In Nokia counters per drop or failure
reason are usually used.
It is understood that Huawei also has
detailed reason counters per failure
and drop.
When running the KPI mapping’s
fine tuning – it would be advisable
to assess performance on the subcounter level (& against the totals) to
understand the actual gaps if any.
Possibly explore HUA KPI changing
as per BT UK case
12
© 2020 Nokia
Nokia internal use
5. VARIOUS NOKIA MATERIAL REFERENCES
FYI
Lessons learned -> LINK
Specific cases:
2G Huawei vs Nokia GSM KPI analysis -> LINK
2G BT UK Border DCR Topic -> LINK
3G Nokia Huawei border interworking -> LINK
4G BT UK single site reference values -> LINK
2G/4G/5G BT UK Cluster Level reference values -> LINK
4G BELL Canada Reference values -> LINK
4G BELL Canada 1st Site Swap Lessons Learned -> LINK
4G Nokia Huawei 4G UL Throughput -> LINK
LTE19B IP Throughput Counters (LTE5702) & impact on throughput-> LINK & Yammer
General information:
EUR NCS Huawei to Nokia Swap KPI REPOSITORY -> LINK
SWAP Do-Don’ts-> LINK
Nokia Performance Benchmarker Trends -> LINK
Nokia Performance Benchmarker Ranks-> LINK
Nokia KPI Reference Report -> LINK
Nokia Discovery -> LINK
13
© 2020 Nokia
Nokia internal use
6. Huawei Reference Documentation
STRICTLY FOR NOKIA INTERNAL USE
Click on the relevant LINK to access the reference source
TECHNOLOGY
# / TOPIC
3G
COMMENT LINK
HUAWEI INFORMATION REPORSITORY
4G
5G
OTHER I
OTHER II
OTHER III
OTHER IV
COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT LINK COMMENT
1
DEFAULT KPI
DEFINITIONS
LINK
LINK
LINK
LINK
n/a
n/a
n/a
n/a
2
COUNTERS CONTROLER
LINK
LINK
n/a
n/a
n/a
n/a
n/a
n/a
COMMON
LINK TRANS / BB /
PLATFORM
LTE EBC (event
LINK based
counters)
n/a
n/a
LINK
LINK
LINK
LINK
LINK
4
PARAMETERS CONTROLER
LINK
LINK
n/a
n/a
n/a
n/a
n/a
LINK
COMMON
LINK TRANS / BB /
PLATFORM
LINK COMMON HW n/a
LINK LTE NBIOT
LINK MULTI MODE
LINK MULTI MODE
n/a
n/a
n/a
PARAMETERS BTS
LINK
LINK
LINK
LINK
6
FEATURES
LINK
LINK PER DOMAIN
LINK FDD
LINK
7
SIGNALLING
FLOWS
n/a
n/a
n/a
n/a
COMMON
RF
SA/NSA/FDD/
LINK LTE TDD
TTD deltas
n/a
…
© 2020 Nokia
Nokia internal use
LTE CLOUD &
DOCUMEN
T TYPE
Default Huawei KPI definitions
(FOLDER)
XLS
XLS
COUNTERS BTS
5
COMMON
RF
NOTES
COUNTER FOLDER
3
…
14
LINK
2G
COMMENT LINK
XLS
PARAMETER FOLDER (Includes
parameter & project
reference documents)
NR CLOUD &
Only the features lists are
provided (FOLDER)
Signaling flows are available under
Huawei's documentation set (HEDEX) for
the counters in question
XLS
XLS
PDF
Huawei to Nokia RAN SWAP
KPI Reference Value Examples
Egidijus Janusevicius
egidijus.janusevicius@nokia.com
2021.01.29
STRICTLY FOR NOKIA INTERNAL USE ONLY
Version 3.0
15
© 2020 Nokia
Nokia internal use
SOURC
E
1. Overseas 4G Swap Reference
Huawei to Nokia Swap
Diligent KPI mapping activity conducted prior
the swap & early swap stages
Ongoing swap KPIs indicates significant
improvement in all metrics.
Reference Information per cluster indicates
good results. Single degraded metrics are
cause by single site/cell issues usually resolved
within defined SLA ranges
16
© 2020 Nokia
Nokia internal use
2. Overseas 4G Swap Reference: Single Site Performance
“Island” Scenario -> Largest Border Exposure Case
Single site was been swapped to Nokia in the middle of the
Huawei cluster.
Swap scenario like this produces largest border exposure.
Most important Service KPIs for Voice & Data have been
improved or maintained respectively.
LTE Data Retainability has been impacted by IWK issue related to
RLF based HO functionality differences, after optimization steps
take it has been taken to the pre-swap levels.
LTE Accessibility has been impacted by SW issue & has been
addressed in extremely timely manner by SW load.
HO reported KPI deltas are mainly due to different counter
pegging implementation on Nokia & Huawei systems.
17
© 2020 Nokia
Nokia internal use
SOURC
E
3.1 European Swap + Modernization Reference Cluster
FYI
SOURC
E
SWAP Cluster of 15 sites
Technologies: 12 2G/4G + 3 2G/4G/5G sites; No 3G sites
SWAP is not “like for like” & is rather a modernization:
-
4G layer has been expanded by adding new carriers. KPI comparison cannot be conducted
-
Traffic layering assumed to be changed due to additional layers. KPI numerical values impacted on
new L800 layer (as coverage layer)
-
3 sites received 5G layer addition
-
No emphasis on the border analysis has been reported. Assumed NO significant issues observed.
-
Local team has reported that during the swap coverage has been increased, what results in some of
the KPI reduction.
-
See next slide for the detailed KPI & Performance data
18
© 2020 Nokia
Nokia internal use
3.2 KPI Reference Data: Pre- vs. Post Modernization
Huawei to Nokia Equipment swap & modernization
2G KPI:
-
Retainability is better
-
Accessibility is on par
4G KPI:
-
Retainability is significantly better
-
Accessibility for Volte is on par
-
Accessibility for Data requires improvement
-
Throughput & Data Volume is significantly better
4G
ERAB SSR%
Nokia
99.86%
Huawei
99.90%
DELTA
-0.04%
DELTA%
-40%
19
LTE SSSR% LTE VOLTE SSR%
99.73%
99.66%
99.83%
99.69%
-0.10%
-0.03%
-59%
-10%
if to calculate as FR%
2G
PS CSSR% Voice CSSR%
Nokia
99.94%
99.79%
Huawei
99.99%
99.80%
DELTA
-0.05%
-0.01%
DELTA%
-500%
-5%
if to calculate as FR%
RRC SSR%
99.88%
99.94%
-0.06%
-100%
PS DCR%
0.80%
1.15%
0.35%
30%
VOICE DCR% HO SR%
0.65%
97.94%
0.79%
97.95%
0.14%
-0.01%
18%
0%
LTE DCR% LTE VOLTE DCR% IFHO SR% HO SR% DL THP kbps UL THP kbps
0.11%
0.21%
99.18% 99.80%
###
791.3
0.25%
0.43%
99.12% 99.83%
###
548.57
0.14%
0.22%
0.06% -0.03%
3804.77
242.73
56%
51%
7%
-18%
###
44%
© 2020 Nokia
Nokia internal use
3.3 KPI Reference Data: Pre- vs. Post Modernization – Site level
Huawei to Nokia Equipment swap & modernization of a cluster of 15 sites in the middle of Huawei
Coverage
NOKIA
KPIs average over the cluster of 15 sites shows performance as good as before for this in the middle of Huawei Coverage.
• A cluster of 15 sites might have around 9 sites in Border with Huawei and around 2 sector over 3 pointing to border so around 18
Sectors over 45 pointing towards Huawei.
 In case strong degradation would have been observed, the average of KPIs would have been worst than before
• Also looking at KPIs Site per Site, most of them are as good as before or low differences which will be investigated. The large
delta are explained by differences in Formulas and event counting in implementation
 Shows that there is no degradation linked to the border effect in 2G and 4G here
20
© 2020 Nokia
Nokia internal use
SOURC
E
4. MEA 4G Swap Reference
Huawei to Nokia Swap
KPI mapping level unknown
Special Area performance achieved or
improved (major highway)
Cluster are KPIs constant
improvement by advanced
optimization resulted in KPIs being on
par
21
© 2020 Nokia
Nokia internal use
SOURC
E
5. MEA 3G/4G NW SWAP REFERENCE
Huawei to Nokia Swap
Very High Density and Quality
Major 4G KPIs show same very high-level
values on the inter-vendor & single
vendor (Nokia) areas.
Major 3G KPIs indicated similar high-level
values on the border & outside the
border area.
KPI mapping level unknown
4G
Border Area
28 Sites
Nokia Area
31 Sites
3G
Border Area
28 Sites
Nokia Area
31 Sites
CSSR
VOLTE
DCR
VOLTE
CSSR
DATA
DCR
DATA
Inter EnodeB
HO SR
Interfreq
HO SR
99,98
0,04
99,84
0,10
99,71
99,04
99,96
0,04
99,87
0,09
99,74
99,43
CSSR
VOICE
MPD
VOICE
CSSR
DATA
SHO
SR
HHO
SR
CSFB
SR
99,87**
4208 *
99,58**
99,98
94,03
99,91
99,92**
4584 *
99,59
99,98
95,47
99,93
* Minutes per Drop for 3G CS : a different way to measure the Drop call. It shows her very good
statistic with a difference of around 8%, meaning for instance if the DCR CS 3G was in Nokia area
at 0,5% , it would be around 0,54% In border area
** - reviewer’s comment: 3G CSSR Values seems to be
too good – considering border impact on the RRC Stage
22
© 2020 Nokia
Nokia internal use
6. European SWAP: “Leopard” scenario
Huawei to Nokia Swap
Absolute KPIs values at good level for Nokia Sites with Only Huawei adjacencies
Single sites swapped to Nokia in the middle of the Huawei area
Red are Nokia Sites : 2G 3G 4G
Green are Huawei Sites : 2G 3G 4G
Deliberate blurring effect applied
23
Voice
Accessibility
Voice Drop
Data Accessibility
Data Drop
Mobility
success rate
rate
2G
98%
1,4%
99,8 %
(low traffic)
3% (low traffic)
98%
3G
99,8%
0,7%
99% (low traffic)
1,4% (low traffic)
98%
4G
99,7%
0,15 %
99,9%
0,4%
99%
KPIs extract: November 2020
© 2020 Nokia
Nokia internal use
JTR IUR Huawei – Nokia (3G)
From outdated report 2018. Assumed no major changes in later stages.
The IOT testing between vendor Nokia RNC (WCDMA 18), and Huawei RNC (SRAN13.1) was
successfully completed and has shown that elements can interwork successfully in the tested
environment.
In addition: Nokia has developed various workaround to tackle any shortcomings related to
RNC border performance (ref. to RAN2972)
Note: RRC Stage Setup Success Rate for CS & PS Services overall Setup KPIs might experience
numerical degradation (usually non end user impacting) due to the UE inter-RNC cell reselections during the RRC setup. See next slide for details.
24
© 2020 Nokia
Nokia internal use
Numerical Impact on CSSR due to cell re-selection during the RRC Setup
E// to Nokia. Similar will apply to Huawei to Nokia
25
© 2020 Nokia
Nokia internal use
THANK YOU TO ALL WHO HAS CONTRIBUTED TO THIS SESSION BY
SHARING VALUABLE INFORMATION
&
ESPECIALLY FOR THE GREAT JOB DONE ON KPI MAPPING ACTIVITIES!!!
Nokia internal use
Nokia internal use
VARIOUS NOKIA MATERIALS REFERENCES: PARAMETER TOPIC
FYI
Parameter & Feature Swap related reference cases:
4G Bell Canada Parameter Mapping -> LINK
4G Bell Canada Single Site Tuning -> LINK
4G Etisalat Volte Parameter Tuning -> LINK
4G BT UK Single site tuning -> LINK
3G JTR IUR Report -> LINK
3G IUR IWK with Huawei (info session)-> LINK
3G IUR Setup Etisalat -> LINK
2G/4G/ 5G BT UK Report -> LINK
General information:
EUR NCS Huawei to Nokia Swap Parameter Repository-> LINK
EUR NCS Huawei to Nokia Swap Feature Repository-> LINK
Nokia Discovery -> LINK
28
© 2020 Nokia
<Document ID: change ID in footer or remove> <Change information classification in footer>
Nokia internal use
Copyright and confidentiality
The contents of this document are proprietary and
confidential property of Nokia. This document is
provided subject to confidentiality obligations of
the applicable agreement(s).
This document is intended for use of Nokia’s
customers and collaborators only for the purpose
for which this document is submitted by Nokia. No
part of this document may be reproduced or made
available to the public or to any third party in any
form or means without the prior written permission
of Nokia. This document is to be used by properly
trained professional personnel. Any use of the
contents in this document is limited strictly to the
use(s) specifically created in the applicable
agreement(s) under which the document is
submitted. The user of this document may
voluntarily provide suggestions, comments or other
feedback to Nokia in respect of the contents of this
document ("Feedback").
29
Such Feedback may be used in Nokia products and
related specifications or other documentation.
Accordingly, if the user of this document gives
Nokia Feedback on the contents of this document,
Nokia may freely use, disclose, reproduce, license,
distribute and otherwise commercialize the
feedback in any Nokia product, technology, service,
specification or other documentation.
are made in relation to the accuracy, reliability or
contents of this document. NOKIA SHALL NOT BE
RESPONSIBLE IN ANY EVENT FOR ERRORS IN THIS
DOCUMENT or for any loss of data or income or
any special, incidental, consequential, indirect or
direct damages howsoever caused, that might arise
from the use of this document or any contents of
this document.
Nokia operates a policy of ongoing development.
Nokia reserves the right to make changes and
improvements to any of the products and/or
services described in this document or withdraw
this document at any time without prior notice.
This document and the product(s) it describes
are protected by copyright according to the
applicable laws.
The contents of this document are provided "as is".
Except as required by applicable law, no warranties
of any kind, either express or implied, including, but
not limited to, the implied warranties of
merchantability and fitness for a particular purpose,
© 2020 Nokia
Nokia internal use
Nokia is a registered trademark of Nokia
Corporation. Other product and company names
mentioned herein may be trademarks or trade
names of their respective owners.