5G RAN2.0 KPI Introduction I ntroduction HUAWEI TECHNOLOGIES CO., LTD. Catalog 1 Basic Concepts of Counters 2 KPI Description 3 5G Counters for NSA Evaluation Evaluation Counter Organization Counter organization: 1. Counter -> Function subset -> Measurement object 2. Counter -> Function subset -> Function set Each counter belongs to only one function subset. Each function subset belongs to only one measurement object and only one function set. Measurement object 1 Function set 1 Function subset 1 Counter 1 Counter 2 Measurement object 2 Function set 2 Function subset 2 Counter 3 … Counter 4 Function set 3 Function subset m Function subset n Counter 5 Counter 6 Counter 7 Counter 8 Function subset l … … Service Measurement Object, Function Set, and Function Subset in 5G RAN2.0 Measurement Object ID Measurement Object Name Measurement of Cell Performance (NRCELL) The NRCELL measurement object contains cell-level function subsets. Cells are deployed on gNodeB central units (CUs). RAN1.0 NRDUCELL Measurement of Local Cell Performance Performance (NRDUCELL (NRDUCELL)) The NRDUCELL measurement object contains NR DU cell function subsets. RAN1.0 gNBDU • Introduced In NRCELL gNodeB • Description The gNodeB measurement object contains Measurement of gNodeB Performance gNodeB-level function subsets, which indicate (gNodeB) the operating status of gNodeBs. RAN1.0 The gNodeB gNodeB measurement object contains gNodeB gNodeB distributio distribution n unit function function subsets, which indicate the operating status of gNodeB distribution unit. RAN2.0 Measurement of gNodeB DU Performance(gNBDU) * In RAN2.0, the NRLOCELL MO is renamed "NRDUCELL" due to configuration object changes. The corresponding function subsets are also renamed. The configuration model of CloudRAN becomes more complicated than that of integrated base stations. Therefore, there are more measurement objects after CloudRAN CloudRAN is introduced. Service Function Subsets in 5G RAN2.0 • Function Subset ID Function Subset Name Function Set Measurement Object Introduced In PDCP.Cell User.Cell Reserve.Cell Thp.DuCell PRB.DuCell RLC.DuCell MAC.DuCell ChMeas.MCS. DuCell ChMeas.CQI. DuCell ChMeas.Pwr. DuCell RA.DuCell DC.Cell Traffic.TRPIP.gNodeB Traffic.TRPIP.gNBDU PDCP Measurement Cell User Quantity Measurement Cell Reserved Counter Measurement Throughput and Data Volume Measurement PRB Measurement RLC Measurement MAC Measurement MCS Measurement CQI Measurement Channel Quality Measurement Random Access Measurement DC Measurement gNodeB Transmission Interface Measurement gNBDU Transmission Interface Measurement PDCP Traffic Reserve Traffic RadioRes RLC MAC ChMeas ChMeas ChMeas RadioRes Algo Transport Transport NRCELL NRCELL NRCELL NRDUCELL NRDUCELL NRDUCELL NRDUCELL NRDUCELL NRDUCELL NRDUCELL NRDUCELL NRDUCELL gNodeB gNBDU RAN1.0 RAN1.0 RAN1.0 RAN1.0 RAN1.0 RAN1.0 RAN1.0 RAN1.0 RAN1.0 RAN1.0 RAN1.0 RAN1.0 RAN2.0 RAN2.0 Algo.Cell Cell Algorithm Measurement Algo NRCELL RAN2.0 Algo.gNodeB CallAtt.gNBDU CallAtt.gNodeB MIMO.DuCell CA.Cell CA.DuCell UlDlDecoupling.Cell gNodeB Algorithm Measurement gNodeB DU Call Processing Times gNodeB Call Processing Times MIMO Channel Measurement Cell CA Measurement DU Cell CA measurement Uplink and Downlink Decoupling Measurement Algo Sig Sig ChMeas Algo Algo Algo gNodeB gNBDU gNodeB NRDUCELL NRDUCELL NRDUCELL NRDUCELL RAN2.0 RAN2.0 RAN2.0 RAN2.0 RAN2.0 RAN2.0 RAN2.0 * In RAN2.0, the NRLOCELL MO is renamed "NRDUCELL" due to configuration object changes. The c orresponding function subsets are 5G Service Counter Report Interval Report Interval A gNodeB reports measurement results to the U2020 at the end of each measurement period. Therefore, the length of a report interval equals the length of measurement period. The measurement period is configurable on the U2020. The length of a long measurement period can be 30 minutes or 60 minutes, and the length of a short measurement period can be 5 minutes or 15 minutes. Only one long measurement period and one short measurement period can be configured. U2000 Performance Management Catalog 1 Basic Concepts of Counters 2 KPI Description 3 5G Counters for NSA Evaluation 5G KPI Architecture Accessibility Retainability In NSA architecture, it is recommended that accessibility KPIs, retainability KPIs, and mobility KPIs be obtained from the LTE side. SA only In SA architecture, 5G-dedicated accessibility KPIS, retainability KPIs, and mobility KPIs will be designed. (will support in 5G RAN2.1) Mobility KPI architecture (counter-based) Service Integrity Utilization • • • • User Uplink/Downlink Average Throughput Cell Uplink/Downlink Average Throughput Uplink/Downlink Resource Block Utilizing Rate Average CPU load SA & NSA Availability Traffic • Radio Network Unavailability Rate • Uplink/Downlink Traffic Volume Average/Maximum User Number • KPI Attributes Object Description Name KPI Formula Associated counter Service Integrity KPIs Service integrity KPIs are used to evaluate the service quality of end-users in the 5G RAN. In RAN2.0, service integrity KPIs include: User Uplink/Downlink Average Throughput Cell Uplink/Downlink Average Throughput Note: Since 5G standards in 3GPP protocols are under discussion, the preceding KPIs may be changed in later versions. Traffic Volume/Throughput Counter Design Data transfer illustration in NSA DC architecture gNodeB eNodeB PDCP PDCP A SCG bearer B SCG split bearer 1 MCG split bearer 1 SCG split bearer 2 RLC MCG split bearer 2 MCG bearer RLC Comparison between RAN measurement and UE measurement MeNB SgNB PDCP traffic volume MCG bearer+MCG split bearer SCG bearer+SCGsplit Cell-specific PDCP traffic volume is not bearer measured. PDCP transmission duration MCG bearer+MCG split bearer SCG bearer+SCGsplit bearer RLC traffic volume MCG bearer+MCG split bearer 2+SCG split bearer 2 MCG bearer+MCG split bearer 2+SCG SCG bearer+MCG split split bearer 2 bearer 1+SCG split (The RLC traffic bearer 1 volume is measured in the PCell in MeNB CA scenarios.) RLC transmission duration MCG bearer+MCG split bearer 2+SCG split bearer 2 SCG bearer+MCG split bearer 1+SCG split bearer 1 MAC traffic volume (MCG bearer+MCG split bearer 2+SCG split bearer 2) Volume of traffic transmitted over the air interface of a specific DU cell (SCG bearer+MCG split bearer 1+SCG split bearer 1) Volume of traffic transmitted over the air interface of a DU cell MAC transmission duration (MCG bearer+MCG split bearer 2+SCG split bearer 2) Duration of transmission over the air interface of a specific DU cell (SCG bearer+MCG split bearer 1+SCG split bearer 1) Duration of transmission over the air interface of a specific DU cell C MAC MAC MAC MAC SCG bearer - option 3A (not supported in versions RAN1.0 and RAN2.0) SCG split bearer - option 3X MCG bearer MCG split bearer - option 3 UE-LTE Cell UE-5G Cell SCG bearer+MCG split bearer 1+SCG split bearer 1 (The RLC traffic volume is measured in the PCell in SeNB CA scenarios.) UE Perception 1. Total traffic volume 2. Service rate. In SA scenarios, the 5QI may not map to the DRB in one-to-one mode. 3. Rates of the following bearers (for PDCP): MCGbearer MCG split bearer SCG bearer SCG split bearer Rates of the following bearers: MCGbearer MCG split bearer 1 SCG split bearer 2 SCG bearer MCG split bearer 2 SCG split bearer 2 (MCG+MCG split bearer 2+SCG split bearer 2) CC-specific traffic volume (SCG bearer+MCG split bearer 1+SCG split bearer 1) CC-specific traffic volume 1. In DC scenarios, the transmit/receive traffic volume and corresponding rate cannot be accurately measured for 5G at the PDCP layer. It is recommended thatUE rates be measured at the RLC layer. 2. In CA scenarios, the rate and traffic volume over the air interface of a specific 5G cell cannot be accurately measured at the RLC layer. It is recommended thatcell rates be measured at the MAC layer Service Integrity KPIs – User Uplink/Downlink Average Throughput These KPIs indicate the average uplink and downlink UE throughputs in a cell. Name User Up link Ave ra ge Throu ghp ut Na me Us er D ownlin k Ave rage T hro ugh put Object Cell or RAN Object Cell or RAN Formula Associated Counter Unit UserULAveThp = UserULRmvSmallPktTrafficVolume/UserULRmvSmallPktTransferTime User Uplink Average Throughput = (N.ThpVol.ULN.ThpVol.UE.UL.SmallPkt)/N.ThpTime.UE.UL.RmvSmallPkt Gbit/s Formula Associated Counter Unit UserDLAveThp = UserDLRmvLastSlotTrafficVolume/UserDLRmvLastSlotTransferTime User Downlink Average Throughput = (N.ThpVol.DL N.ThpVol.DL.LastSlot)/N.ThpTime.DL.RmvLastSlot Gbit/s RAN2.0 RAN1.0 Name User Uplink Average Throughput Name User Downlink Average Throughput Object Cell or RAN Object Cell or RAN Formula Associated Counter Unit UserULAveThp = UserULTrafficVolume/UserULTransferTime User Uplink Average Throughput = N.ThpVol.UL/N.ThpTime.UL Gbps Formula Associated Counter Unit UserDLAveThp = UserDLTrafficVolume/UserDLTransferTime User Downlink Average Throughput = N.ThpVol.DL/N.ThpTime.DL Gbps Service Integrity KPIs – Cell Uplink/Downlink Average Throughput These KPIs indicate the average uplink and downlink cell throughputs, which reflect the cell capacity. Name Cell Uplink Average Throughput Name Cell Downlink Average Throughput Object Cell or RAN Object Cell or RAN Formula CellULAveThp = CellULTrafficVolume/CellULTransferTime Formula Cell Downlink Average Throughput = N.ThpVol.DL.Cell/N.ThpTime.DL.Cell Cell Uplink Average Throughput = N.ThpVol.UL.Cell/N.ThpTime.UL.Cell Associated Counter Note: Unit Gbit/s When comparing this KPI with the theoretical peak rate for TDD, pay attention to the uplink-downlink subframe configuration. CellDLAveThp = CellDLTrafficVolume/CellDLTransferTime Associated Counter Note: Unit Gbit/s Note: The throughputs are measured based on MAC TBs. When comparing this KPI with the theoretical peak rate for TDD, pay attention to the uplink-downlink subframe configuration. Utilization KPIs Utilization KPIs are used to evaluate the capabilities, such as the capability to meet traffic demands, in specific internal conditions. In RAN2.0, utilization KPIs include: Uplink/Downlink Resource Block Utilizing Rate Average CPU Load Note: Since 5G standards in 3GPP protocols are under discussion, the preceding KPIs may be changed in later versions. Utilization KPIs – Uplink/Downlink Resource Block Utilizing Rate These KPIs indicate the busy-hour uplink and downlink RB usages in a cell or RAN. Name Uplink Resource Block Utilizing Rate Name Downlink Resource Block Utilizing Rate Object Cell or RAN Object Cell or RAN Formula RB_URUL = (RB_UsedUL/RB_AvailableUL) x 100% Formula RB_URDL = (RB_UsedDL/RB_AvailableDL) x 100% Associated Counter Uplink Resource Block Utilizing Rate = (N.PRB.UL.Used.Avg/N.PRB.UL.Avail.Avg) x 1 00% Associated Counter Downlink Resource Block Utilizing Rate = (N.PRB.DL.Used.Avg/N.PRB.DL.Avail.Avg) x 10 0% Unit % Unit % Utilization KPI – Average CPU Load This KPI indicates the CPU usage during busy hours. Name Average CPU Load Object CPU Formula Associated Counter Unit MeanCPUUtility Average CPU Load = VS.BBUBoard.CPULoad.Mean % Availability KPI An available cell indicates that it can provide EPS bearer services. In RAN1.0, availability KPI is not supported. In RAN2.0, availability KPI includes: Radio network unavailability rate Availability KPI – Radio Network Unavailability Rate This KPI indicates the percentage of time when cells in a radio network are unavailable. It is used to evaluate the deterioration of network performance caused by cell unavailability of the radio network during busy hours. Name Rad io Ne twork Un availa bility Ra te Object Radio Network Formula Associated Counter Unit RAN_Unavail_Rate = (ΣCellUnavailTime /(TheTotalNumberOfCellsInCluster x {SP} x 60)) x 100% Radio Network Unavailability Rate =((N.Cell.Unavail.Dur.System + N.Cell.Unavail.Dur.Manual)/(Number of cells x {SP} x 60)) x 100% SP represents the reporting period of counters. The unit is minute. % Traffic KPIs Traffic KPIs are used to measure the traffic volume on the 5G RAN. In RAN2.0, utilization KPIs include: Uplink/Downlink Traffic Volume Average/Maximum User Number Note: Since 5G standards in 3GPP protocols are under discussion, the preceding KPIs may be changed in later versions. Traffic KPIs – Uplink/Downlink Traffic Volume These KPIs indicate the uplink and downlink traffic volumes in a cell, which are measured at the Radio Link Control (RLC) layer. Na me Uplink Traffic Volume Na me Down link Tra ffic Volume Object Cell or RAN Object Cell or RAN Formula Associated Counter Unit ULTrafficVolume Uplink Traffic Volume = N.ThpVol.UL kbit F ormula Associated Counter Unit DLTrafficVo lume Downlink Traffic Volume = N.ThpVol.DL kbit Traffic KPI – Average/Maximum User Number This KPI indicates the average number of UEs in RRC_Connected mode in a cell. The gNodeB samples and records the number of UEs every second and then calculates the average value of these samples at the end of each measurement period. Name Average User Number Name Maximum User Number Object Cell or RAN Object Cell or RAN Formula Associated Counter Unit AvgUserNumber Average User Number = N.User.RRCConn.Avg N/A Formula Associated Counter Unit MaxUserNumber Maximum User Number = N.User.RRCConn.Max N/A Catalog 1 Basic Concepts of Counters 2 KPI Description 3 5G Counters for NSA Evaluation RAN2.0 NSA Architecture and SgNB Evaluation NSA Evaluation NSA Architecture • • Control plane: LTE User plane: GBR services: LTE Non-GBR services: LTE and NR, controlled by a specific algorithm EPC S1-C S1-U LTE eNodeB * * SgNB addition • SgNB access success rate SgNB release • SgNB abnormal release rate SgNB modification • SgNB PSCell change success rate • Number of NSA UEs NSA option 3X: Total volume of PDCP traffic Volume of PDCP traffic transferred to MeNB S1-U gNodeB Data split from SgNB in option 3X Data split from MeNB in option 3. Traffic evaluation • SgNB Addition Procedure and Counters UE MN SN S-GW MME Counter Name Counter Description Measurement Point 1. SgNB Addition Request 3. RRCConnectionRecon figuration 2. SgNB Addition Request Acknowledge A 4. RRCConnectionRecon figurationComplete 5. SgNB Reconfiguration Complete Number of SgNB N.NsaDc.S addition gNB.Add.A requests in tt the LTE-NR NSA DC scenario As shown at point A in the figure, the N.NsaDc.SgNB.Add.Att counter is incremented by 1 each time the gNodeB receives an SgNB Addition Request message from the eNodeB. The counter value is accumulated in the PSCell specified by the gNodeB. Number of successful N.NsaDc.S SgNB gNB.Add.S additions in ucc the LTE-NR NSA DC scenario As shown at point B in the figure, the N.NsaDc.SgNB.Add.Succ counter is incremented by 1 each time the gNodeB receives an SgNB Reconfiguration Complete message from the eNodeB. The counter value is accumulated in the PSCell specified by the gNodeB. 6. Random Access Procedure 7. SN Status Transfer 8. Data Forwarding Path Update procedure B 9. E-RAB Modification Indication 10. Bearer Modication 11. End Marker Packet 12. E-RAB Modification Confirmation SgNB Release Procedure and Counters UE MN 3. RRCConnectionRecon figuration 4. RRCConnectionRecon figurationComplete SN 1. SgNB Release Request 2. SgNB Release Request Acknowledge S-GW MME A 5. SN Status Transfer 7. Secondary RAT Data Volume Report 8. Path Update procedure 9. UE Context Release 3. RRCConnectionRecon figuration MN 1. SgNB Release Required 2. SgNB Release Confirm 4. RRCConnectionRecon figurationComplete SN S-GW A B 5. SN Status Transfer 6. Data Forwarding 7. Secondary RAT Data Volume report 8. Path Update procedure 9. UE Context Release Counter Description Measurement Point As shown at point A in figure 1, the N.NsaDc.SgNB.Rel counter is incremented by 1 each time the gNodeB receives an Total numb er SgNB Release Request message from the of SgNB eNodeB. As shown at point B in figure 2, N.NsaDc. releases in the the N.NsaDc.SgNB.Rel counter is SgNB.Rel LTE-NR NSA incremented by 1 each time the gNodeB DC scenario receives an SgNB Release Confirm message from the eNodeB. The counter value is accumulated in the PSCell of LTENR NSA DC UEs. 6. Data Forwarding UE Counter Name MME As shown at points A and B in figure 2, after the gNodeB sends an SgNB Release Required message with the cause value of Number of "Radio Connection With UE Lost" or abnormal N.NsaDc. "Failure in the Radio Interface" to the SgNB SgNB.Abn eNodeB, if the gNodeB receives an SgNB releases in the ormRel Release Confirm message from the LTE-NR NSA eNodeB, the N.NsaDc.SgNB.AbnormRel DC scenario counter is incremented by 1. The counter value is accumulated in the PSCell of LTENR NSA DC UEs. SgNB Modification Procedure and Counters Figure 1 Change to an intra-site gNodeB Counter Name A Number of intraN.NsaDc.In SgNB PSCell traSgNB.P change requests SCell.Chan in the LTE-NR ge.Att NSA DC scenario B N.NsaDc.In traSgNB.P SCell.Chan ge.Succ Figure 2 Change to inter-site gNodeB UE MN S-SN 1. SgNB Change Required 4. RRCConnectionRecon figuration 2. SgNB Addition Request T-SN S-GW A 3. SgNB Addition Request Acknowledge 5. RRCConnectionRecon figurationComplete 6. SgNB Change Confirm 7. SgNB Reconfiguration Complete 8. Random Access Procedure B 9a. SN Status Transfer 9b. SN Status Transfer 10. Data Forwarding 11. Secondary RAT Data Volume Report 12. E-RAB Modification Indication 13. Bearer Modification 14. End Marker Packet 16. E-RAB Modification Confirm 15. New Path Counter Description MME Number of successful intraSgNB PSCell changes in the LTE-NR NSA DC scenario Measurement Point As shown at point A in figure 1, the N.NsaDc.IntraSgNB.PSCell.Change.Att counter is incremented by 1 each time the gNodeB sends an SgNB Modification Required message to the eNodeB to request a PSCell change. As shown at point B in figure 1, the N.NsaDc.IntraSgNB.PSCell.Change.Succ counter is incremented by 1 each time the gNodeB is notified of a PSCell change via an SgNB Modification Confirm message from the eNodeB. Number of interN.NsaDc.In SgNB PSCell terSgNB.P change requests SCell.Chan in the LTE-NR ge.Att NSA DC scenario As shown at point A in figure 2, the N.NsaDc.InterSgNB.PSCell.Change.Att counter is incremented by 1 each time the gNodeB sends an SgNB Change Required message to the eNodeB. The counter value is accumulated in the PSCell of the LTE-NR NSA DC UE. Number of N.NsaDc.In successful interterSgNB.P SgNB PSCell SCell.Chan changes in the ge.Succ LTE-NR NSA DC scenario As shown at point B in figure 2, the N.NsaDc.InterSgNB.PSCell.Change.Succ counter is incremented by 1 each time the gNodeB receives an SgNB Change Confirm message from the eNodeB. The counter value is accumulated in the PSCell of the LTE-NR NSA DC UE. Number of SgNB DRB Additions Figure 1 Counter Name Counter Description Measurement Point As shown at point A in figure 1, when the gNodeB receives an SgNB Addition Request message from the eNodeB, the gNodeB adds the value of E-RAB number contained in "ENumber of DRB RABs To Be Added List" to the addition requests N.NsaDc.D N.NsaDc.DRB.Add.Att counter. for LTE-NR NSA RB.Add.Att As shown at point A in figure 2, when the DC UEs on the gNodeB receives an SgNB Modification SgNB Request message from the eNodeB, the gNodeB adds the value of E-RAB number contained in "E-RABs To Be Added List" to the N.NsaDc.DRB.Add.Att counter. Figure 2 Number of successful DRB N.NsaDc.D additions for RB.Add.Su LTE-NR NSA DC cc UEs on the SgNB As shown at points B and C in figure 1, when the gNodeB receives an SgNB Reconfiguration Complete message from the eNodeB, the gNodeB adds the value of E-RAB number contained in "E-RABs Admitted To Be Added Item" of the SgNB Addition Request Acknowledge message to the N.NsaDc.DRB.Add.Succ counter. As shown at points B and C in figure 2, when the gNodeB receives an SgNB Reconfiguration Complete message from the eNodeB, the gNodeB adds the value of E-RAB number contained in "E-RABs Admitted To Be Added Item" of the SgNB Modification Request Acknowledge message to the Number of SgNB DRB Releases Figure 1 Counter Name Counter Description Figure 2 Number of DRB N.NsaDc.DRB releases for LTE-NR .Rel NSA DC UEs on the SgNB Figure 3 Figure 4 Number of abnormal N.NsaDc.DRB DRB releases for .AbnormRel LTE-NR NSA DC UEs on the SgNB Measurement Point As shown at points A and B in figure 1, when the gNodeB receives an SgNB Modification Confirm message from the eNodeB, the gNodeB adds the value of E-RAB number in "E-RABs To Be Released Item" of the SgNB Modification Required message to the N.NsaDc.DRB.Rel counter. As shown at point B in figure 2, when the gNodeB receives an SgNB Release Confirm message from the eNodeB, the gNodeB adds the value of E-RAB number contained in "E-RABs To Be Released Item" to the N.NsaDc.DRB.Rel counter. As shown at points B and C in figure 3, when the gNodeB receives an SgNB Reconfiguration Complete message from the eNodeB, the gNodeB adds the value of E-RAB number contained in "E-RABs Admitted To Be Released Item" of the SgNB Modification Request Acknowledge message to the N.NsaDc.DRB.Rel counter. As shown at point A in figure 4, when the gNodeB receives an SgNB Release Request message from the eNodeB, the gNodeB adds the value of E-RAB number contained in "E-RABs To Be Released Item" to the N.NsaDc.DRB.Rel counter. The counter value is accumulated in the PSCell of LTE-NR NSA DC UEs. As shown at points A and B in figure 1, when the gNodeB receives an SgNB Modification Confirm message from the eNodeB, the gNodeB adds the value of E-RAB number contained in "E-RABs To Be Released Item" to the N.NsaDc.DRB.AbnormRel counter if the cause value contained in the SgNB Modification Required message is "Radio Connection With UE Lost" or "Failure in the Radio Interface". As shown at points A and B in figure 2, when the gNodeB receives an SgNB Release Confirm message from the eNodeB, the gNodeB adds the value of ERAB number contained in "E-RABs To Be Released Item" to the N.NsaDc.DRB.AbnormRel counter if the cause value contained in the SgNB Release Required message is "Radio Connection With UE Lost" or "Failure in SgNB Traffic Evaluation In NSA architecture, traffic volume is evaluated using the counters listed in the following table. N.PDCP.UL.TrfSDU.TxPackets N.PDCP.DL.TrfSDU.RxPackets N.PDCP.Vol.UL.TrfSDU.Tx N.PDCP.Vol.DL.TrfSDU.Rx N.PDCP.DL.TrfPDU.TxPackets N.PDCP.UL.TrfPDU.RxPackets Number of PDCP SDUs for DRBs transmi tted to the upper layer in a cell Number of SDUs for DRBs received at the P DCP layer in a cell Traffic of PDCP SDUs for DRBs transmitted to the upper layer in a cell Traffic of SDUs for DRBs received at the PDCP layer in a cell Number of PDCP PDUs for DRBs transmi tted to the RLC layer in a cell Number of RLC PDUs for DRBs received at the PDCP layer in a cell N.PDCP.DL.TrfPDU.ReTxPackets Number of PDCP PDUs retransmitted to the RLC layer i n a cell N.PDCP.Vol.DL.TrfPDU.Tx Traffic of PDCP PDUs for DRBs transmitted to the RLC layer in a cell N.PDCP.Vol.UL.TrfPDU.Rx Traffic of RLC PDUs for DRBs received at the PDCP layer in a cell N.PDCP.UL.TrfSDU.RxPacket.Loss Number of lost uplink PDCP SDUs for DRBs in a cell Number of PDCP PDUs transmitted in the downlink over the X2 interface in a cell Number of PDCP PDUs for DRBs received i n the uplink over N.PDCP.UL.X2U.TrfPDU.RxPackets the X2 Number of PDCP PDUs retransmitted in the downlink over the N.PDCP.DL.X2U.ReqRetransPackets X2 interface in a cell Traffic of PDCP PDUs for DRBs transmitted in the downlink N.PDCP.Vol.DL.X2U.TrfPDU.Tx over the X2 interface in a cell Traffic of PDCP PDUs for DRBs received in the uplink over the N.PDCP.Vol.UL.X2U.TrfPDU.Rx X2 interface in a cell N.PDCP.DL.X2U.TrfPDU.TxPackets Number of SDUs transmitted to or received from the upper layer and the volume of the SDUs Uplink PDCP packet loss Number of PDUs transmitted to or received from the lower layer and the volume of the PDUs Number of PDUs transmitted to LTE and the volume of the PDUs Downlink air interface packet loss In the NSA architecture, the average/maximum user number of NSA DC is evaluated using the N.User.NsaDc.PSCell.Avg counter. In the NSA architecture, the number of RRC users is evaluated using the N.User.RRCConn.Avg counter and the N.User.RRCConn.Max counter. These two counters are also applicable to SA Thank you www.huawei.com Copyright © 2018 Huawei Technologies Co., Ltd. All Rights Reserved. The information in this document may contain predictive statements including, without limitation, statements regarding the future financial and operating results, future product portfolio, new technology, etc. There are a number of factors that could cause actual results and developments to differ materially from those expressed or implied in the predictive statements. Therefore, such information is provided for reference purpose only and constitutes neither an offer nor an acceptance. Huawei may change the information at any time without notice. LTE: NSA DC Feature Evaluation NSA DC MeNB Evaluation NSA Architecture • • Control plane: LTE User plane: GBR services: LTE Non-GBR services: LTE and NR, controlled by a specific algorithm EPC S1-C S1-U LTE eNodeB * * SgNB addition • SgNB access success rate SgNB release • SgNB abnormal release rate SgNB modification • SgNB PSCell change success rate • Number of NSA UEs NSA option 3X: Total volume of PDCP traffic Volume of PDCP traffic transferred to MeNB S1-U gNodeB Data split from SgNB in option 3X Data split from MeNB in option 3. Traffic evaluation • SgNB Addition Procedure and Counters UE MeNB SgNB Counter Name SgNB Addition Request Acknowledge L.NsaDc.SgNB. Add.Att As shown at point A in figure 1, the L.NsaDc.SgNB.Add.Att counter is incremented each time the eNodeB sends an SgNB Addition Request message to the gNodeB. L.NsaDc.SgNB. Add.Succ Total number of successful SgNB additions for UEs that treat the local cell as their PCell in the LTE-NR NSA DC state As shown at point B in figure 1, the L.NsaDc.SgNB.Add.Succ counter is incremented each time the eNodeB sends an SgNB Reconfiguration Complete message to the gNodeB. RRCConnectionReconfiguration RRCConnectionReconfiguration Complete SgNB Reconfiguration Complete B Measurement Point Total number of SgNB addition attempts for UEs that treat the local cell as their PCell in the LTE-NR NSA DC state SgNB Addition Request A Counter Description SgNB Release Procedure and Counters Figure 1 UE MeNB SgNB Counter Name SgNB Release Request Counter Description A RRCConnectionReconfiguration RRCConnectionReconfiguration Complete L.NsaDc.SgNB. Rmv.Att Figure 2 UE MeNB SgNB SgNB Release Required SgNB Release Confirm A RRCConnectionReconfiguration RRCConnectionReconfiguration Complete Total number of SgNB removal attempts for UEs that treat the local cell as their PCell in the LTE-NR NSA DC state Measurement Point As shown at point A in figure 1, the L.NsaDc.SgNB.Rmv.Att counter is incremented each time the eNodeB sends an SgNB Release Request message to the gNodeB. As shown at point A in figure 2, the L.NsaDc.SgNB.Rmv.Att counter is incremented each time the eNodeB sends an SgNB Release Confirm message to the gNodeB. SgNB Modification Procedure and Counters Figure 1 UE Counter Name MeNB S-SgNB Counter Description Measurement Point Total number of SCG change attempts for UEs that treat the local cell as their PCell in the LTE-NR NSA DC state As shown at point A in figure 1, the L.NsaDc.SCG.Change.Att counter is incremented each time the eNodeB receives an SgNB Change Required message from the gNodeB. As shown at point A in figure 2, the L.NsaDc.SCG.Change.Att counter is incremented each time the eNodeB sends an SgNB Modification Request message to the gNodeB. As shown at point A in figure 4, the L.NsaDc.SCG.Change.Att counter is incremented each time the eNodeB receives an SgNB Modification Required message from the gNodeB. The counter is incremented in the PCell of UEs in the LTE-NR NSA DC state. Total number of successful SCG changes for UEs that treat the local cell as their PCell in the LTE-NR NSA DC state As shown at point B in figure 2, the L.NsaDc.SCG.Change.Succ counter is incremented each time the eNodeB sends an SgNB Change Confirm message to the g NodeB. As shown at point B in figure 3, the L.NsaDc.SCG.Change.Succ counter is incremented each time the eNodeB sends an SgNB Reconfiguration Complete message to the gNodeB. As shown at point B in figure 4, the L.NsaDc.SCG.Change.Succ counter is incremented each time the eNodeB sends an SgNB Modification Confirm message to the gNodeB. The counter is incremented in the PCell of UEs in the LTE-NR NSA DC state. T-SgNB SgNB Change Required A SgNB Addition Request SgNB Addition Request Acknowledge RRCConnectionReconfiguration RRCConnectionReconfiguration Complete L.NsaDc.SCG. Change.Att SgNB Change Confirm B Figure 2 UE MeNB SgNB SgNB Modification Request A SgNB Modification Request Acknowledge RRCConnectionReconfiguration RRCConnectionReconfiguration Complete SgNB Reconfiguration Complete B L.NsaDc.SCG. Change.Succ Figure 3 UE MeNB SgNB SgNB Modification Required A RRCConnectionReconfiguration RRCConnectionReconfiguration Complete SgNB Modification Confirm Number of PCell Change Executions and Successful PCell Changes in the LTE-NR NSA DC State (Newly Added in RAN2.0) Figure 1 Counter ID Counter Name Counter Description Measurement Point As shown at point A in figure 1, L.NsaDc.PCell.Change.Exec counter is incremented each time the source MeNB sends an RRCConnectionReconfiguration message containing the configuration indicator of the SgNB handover to UEs. As shown at point B in figure 1, L.NsaDc.PCell.Change.Succ counter is incremented each time the source MeNB receives an UE Context Release message from the target MeNB if the RRCConnectionReconfiguration message contains the configuration indicator of the SgNB handover at point B. As shown at point A in figure 2, Total number L.NsaDc.PCell.Change.Exec is incremented each time of successful the source MeNB sends an L.NsaDc.PCell.Ch PCell changes ange.Succ RRCConnectionReconfiguration message containing in the LTE-NR NSA DC state. the configuration indicator of the SgNB handover to UEs. As shown at point B in figure 3, L.NsaDc.PCell.Change.Succ is incremented each time the source MeNB receives an RRCConnectionReconfigurationComplete message from UE. The counter value is accumulated in the PCell of the LTE-NR NSA DC UE. Total number of PCell L.NsaDc.PCell.Ch change ange.Exec executions in the LTE-NR NSA DC state. Figure 2 Successful MeNB changes is defined as follows: Change with SgNB. That is, the SgNB remains the same after MeNB changes. Total Number of Abnormal E-RAB Releases for NSA DC UEs (Newly Added in RAN2.0) Counter ID Counter Name Counter Description Total number L.NsaDc.E- of abnormal RAB.Abnorm E-RAB Rel releases in the LTE-NR NSA DC state. Measurement Point As shown at point A in this figure, L.NsaDc.E-RAB.AbnormRel is incremented each time MeNB sends an E-RAB RELEASE INDICATION message to MME. The counter value is accumulated if the corresponding bearer has data transmission and the release cause is not normal release, detach, user inactivity, Om-intervention, CS fallback triggered, UE not available for PS service and inter-RAT redirection. If the E-RAB RELEASE INDICATION message requires to release multiple ERABs at the same t ime, the counter value is accumulated according to the number of ERAB. Number of SCG-Related Failures Counter ID UE Counter Name Counter Description EUTRAN RRC connection reconfiguration (Scg-Configuration) 1526747855 SCGFailureInformation Total number of SCGrelated failures for UEs L.NsaDc.ScgFail that treat the local cell as ure their PCell in the LTE-NR NSA DC state A 4 scenarios will be counted as SCG Failure. I. SCG RLF, II. SN change failure, III. SCG configuration failure (only for messages on SRB3), IV. SCG RRC integrity check failure (on SRB3) , Measurement Point As shown at point A in this figure, the L.NsaDc.ScgFailure counter is incremented each time the eNodeB receives an SCGFailureInformation message from the UE. The counter value is accumulated in the PCell of the LTE-NR NSA DC UE. Traffic Volume and User Number Counter ID Counter Name Counter Description Measurement Point L.Thpt.bits.DL.McgSplit.MeNB Total traffic volume offloaded from UEs in the PCell at the PDCP layer to the MeNB during LTE-NR NSA DC Option3 offloading The counter measures the total downlink traffic volume offloaded from LTE-NR NSA DC UEs in the cell at the PDCP layer to the MeNB during Option3 offloading. The volume of successfully offloaded SDU data is accumulated as the value of this counter. L.Thpt.bits.DL.McgSplit.SgNB Total traffic volume offloaded from UEs in the PCell at the PDCP layer to the SgNB during LTE-NR NSA DC Option3 offloading The counter measures the total downlink traffic volume offloaded from LTE-NR NSA DC UEs in the cell at the PDCP layer to the SgNB during Option3 offloading. The volume of successfully offloaded SDU data is accumulated as the value of this counter. L.Thpt.bits.UL.McgSplit.MeNB Total uplink MeNB traffic volume received by UEs in the PCell from the PDCP layer of the MeNB during LTE-NR NSA DC Option3 offloading The counter measures the total uplink traffic volume received at the PDCP layer by LTE-NR NSA DC UEs from the MeNB during uplink Option3 offloading. The volume of successfully received SDU data is accumulated as the value of this counter. L.Thpt.bits.UL.McgSplit.SgNB Total uplink SgNB traffic volume received by UEs in the PCell from the PDCP layer of the MeNB during LTE-NR NSA DC Option3 offloading The counter measures the total uplink traffic volume received at the PDCP layer by LTE-NR NSA DC UEs from the SgNB during uplink Option3 offloading. The volume of successfully received SDU data is accumulated as the value of this counter. Average number of UEs that treat the local L.Traffic.User.NsaDc.PCell.Avg cell as their PCell in the LTE-NR NSA DC state The number of all UEs in connected mode (in both the LTE PCell and the NR PSCell) that are in the LTE-NR NSA DC state and treat the local cell as their PCell are sampled per second in a cell. At the end of a measurement period, the average of these sampling results is taken as the counter value. 3GPP PM Protocol Roadmap • 3GPP has discussed E2E KPI in SA5. Slicing and Impacted existing TS/TR latency is to be updated in June 2018. The UE TS/TR No. throughput depending on RAN2 has not been discussed yet. • 32.425 Add performance measurements for ng-eNB in terms of connectivity with 5GC SA#80 (Jun 2018) 32.426 Add performance measurements for EPC in terms of connectivity with NR SA#80 (Jun 2018) 32.450 Add KPIs for ng-eNB in terms of connectivity with 5GC SA#80 (Jun 2018) 32.451 Add KPI requirements for ng-eNB in terms of connectivity with 5GC SA#80 (Jun 2018) 32.455 Add KPIs for EPC in terms of connectivity with NR SA#80 (Jun 2018) The RAN KPI (Layer 2 Measurements) is to be discussed in the RAN2, including the UE throughput. The RAN2 mainly deals with t he NSA/SA basic protocol and has not been planned for discussion. Target completion plenary# Description of change New specifications Series Title For info at TSG# For approval at TSG# "28.XXX" Performance Management for 5G networks and network slicing; stage 1 SA#79 (Mar 2018) SA#80 (Jun 2018) "28.XXX" Performance Management for 5G networks and network slicing; stage 2 and stage 3 SA#79 (Mar 2018) SA#80 (Jun 2018) "28.XXX" Performance measurements and assurance data for NG-RAN Network Functions SA#79 (Mar 2018) SA#80 (Jun 2018) "28.XXX" Performance measurements and assurance data for 5GC Network Functions SA#79 (Mar 2018) SA#80 (Jun 2018) "28.XXX" End to end KPIs, Performance measurements and assurance data for 5G networks and network slicing SA#79 (Mar 2018) SA#80 (Jun 2018)