LTE L11 Throughput Troubleshooting Techniques Introduction Why learn about Throughput Troubleshooting › LTE provides data, lots of data › Throughput is shared in time and frequency › Users notice throughput problems › Learn to troubleshoot the LTE RAN for throughput problems › Learn to isolate the domain causing throughput degradation Scope and objectives Scope › RBS Initial Checks › Radio Analysis › Transport Analysis › End-2-End Analysis Objectives › Isolate throughput problems into domains › Pinpoint causes of throughput degradation clearly within domains through theory, traces and practical examples > Overview Agenda 1. Overview 2. Initial Checks 3. Radio Analysis 4. Transport Analysis 5. E2E Analysis LTE RBS User plane Overview LTE RBS User Plane Overview › User plane visualisation › Network assumptions › L11 software limitations User Plane Domains S6a HSS MME S1-MME S11 FTP Server eNB UE S1 Uu IP Backbone S-GW S1-U S5/S8 P-GW Internet FTP Serv. FTP Client IP MAC GTP-U UDP IP Data Link GTP-U GTP-U UDP UDP IP IP Data Link Data Link IP GTP-U UDP IP Data Link Physical Physical Physical Physical PDCP PDCP RLC RLC MAC Physical Uu Radio Domain Relay Relay Physical S1-U S5/S8 Transport Domain e2e Domain SGi IP Data Flow over Air (RBS/UE) PDCP Header Payload Header Header Payload PDCP SDU Header Header Payload Header Payload Payload PDCP Header PDCP Header PDCP Header PDCP PDU ARQ RLC SDU RLC Payload RLC Header RLC Header RLC Header PHY MAC RLC PDU MAC Header MAC Header HARQ MAC PDU Transport Block CRC Transport Block CRC network assumptions › Network configuration and integration is complete › Data sessions have been previously verified › Basic RBS troubleshooting has been performed: – Node alarms verified – MO status – Cell availability › See the ?Fundamental RBS Troubleshooting Techniques? for further checks that can be performed Release Limitations › L11A contains some limitations that directly affect end user throughput – One SE per TTI in UL and DL (in L11A GA) › Each cell is treated individually, so there could be up to 3 users simultaneously in an eNB › SIB is scheduled the same as user data, so nothing can be scheduled at the same time as SIB – DUL user plane capacity limited to 150 Mbps (20MHz) – 100PRBs UL, 150 PRBs DL. – 16QAM UL (up to MCS24) – MCS28 disabled in DL by default (requires CFI=1 also) – Fixed CFI (number of OFDM symbols for PDCCH) › Default is CFI=2 for 5MHz and less. › CFI=3 not possible for >= 10MHz due to system limitations, this reduces PDCCH scheduling opportunities. Initial Checks Initial checks › For throughput issues, some essential checks are required: 1. 2. 3. 4. 5. 6. Network changes & Basic troubleshooting PC/Server settings UE categories UE subscriber profile RBS parameters Enabled features NW Changes and Basic Troubleshooting › Network/node changes can affect network throughput › Some common examples include: – Network configuration changes (e.g. adding/changing/removing hardware) – RBS parameter changes (all MOs under ENodeBFunction, system constants, EricssonOnly hidden parameters, e.g. DataRadioBearer) – IP address plan changes – Transport Network changes (add/reduce capacity on TN) – DNS updates – Hint: › To see RBS level changes (MOs/parameters): Moshell> lgo › To capture detailed RBS level logs: Moshell> dcg › Basic troubleshooting checks include: – Alarm, event and system log checks – MO health status PC/Server Settings › Determine the applications or tools used in testing/monitoring throughput › Confirm the end user PC settings: – Laptop specification can impact throughput (processors, memory, USB bus, HDD speed, plugged into AC power, etc) – MTU settings in PC (1360 optimal for eNB in L11A to prevent fragmentation) – Throughput monitors (e.g. Netpersec, only good for downlink UDP measurements, uplink must be measured at receiving side for UDP) – TCP enhancements in Vista (experimental), Vista should “auto-tune”. › Confirm server settings: – – – – FTP server configuration Linux TCP setting/guide iperf (UDP & TCP) – be sure to use packet size 1360 for UDP (not default 1470). Always check first with UDP rather than TCP, as UDP is less prone to display problems as a result of jitter variations and packet loss. UE Categories › The UE Category limits throughput possibilities › 5 UE Categories are defined in 3GPP TS 36.306 › The UE-Cat is sent in the UE Capability Transfer procedure (RRC UECapabilityInformation) › The COLI ue command provides detailed capability info (KO) for connected UEs UE Category DL Maximum number of DL-SCH transport block bits received within a TTI Maximum number of bits Total number of of a DL-SCH transport soft channel block received within a bits TTI Maximum number of supported layers for spatial multiplexing in DL Category 1 10296 10296 250368 1 Category 2 51024 51024 1237248 2 Category 3 102048 75376 1237248 2 Category 4 150752 75376 1827072 2 Category 5 299552 149776 3667200 4 UE Category UL Maximum number of bits Support for 64QAM of an UL-SCH in UL transport block transmitted within a TTI Category 1 5160 No Category 2 25456 No Category 3 51024 No Category 4 51024 No Category 5 75376 Yes UE Subscriber profile › End User (EPS User) subscription data is stored in the HSS › The EPS User Profile data is identified by its IMSI number › The profile consists of: – – – – – MSISDN number Operator Determined Barring (ODB) APN Operator Identifier Replacement Subscribed Charging Characteristics Aggregate Maximum Bit Rate (AMBR) › Max requested bandwidth in Downlink › Max requested bandwidth in Uplink – RAT frequency selection priority – APN configuration profile: › Default Context Identifier (default APN for the EPS User) › APN Configuration (every APN associated to the EPS User) RBS Parameters RN › RN MO parameters: – EUtranCellFDD › dlChannelBandwidth / ulChannelBandwidth › (nrOfSymbolsPdcch) (Control Region Size) NOTE: currently controlled by SC38 in L11A › noOfUsedTxAntennas controls whether OLSM MIMO is used (2) or not. › partOfRadioPower NOTE: this is the % part of RU capability independent of SectorEquipmentFunction::confOutputPower settings › pZeroNominalPucch some UEs need this to be increased or ACK/NACKs are not received successfully on PUCCH. › pZeroNominalPusch some UEs need this to be increased from default or lots of errors seen on PUSCH – SectorEquipmentFunction=Sx › confOutputPower / fqBand (readOnly) – DataRadioBearer › Various parameters for RLC status reporting and retransmission. Should be set to recommended values. – MACConfiguration › xxMaxHARQTx – enable (>1) or disable (1) HARQ. Recommended to use 4 HARQTx. › tPeriodicBSRTimer – seen in UE testing to have some impact, recommend to set to 5ms. › tTimeAlignmentTimer – seen in UE testing to have some impact, recommend to set to 5120ms RBS PARAMTERS TN › TN MO parameters: – GigabitEthernet=1 › actualSpeedDuplex – if you see half-duplex, it could be a problem with autonegotiation › dscpPbitMap (QoS mapping from L3 to L2) – IpInterface=2 (rec. MO id for Signalling and Payload) › vLan/vid (true/false and vlan id) – IpAccessHostEt=1 › ipAddress (X2/S1 control/user plane termination) – IpSyncRef (if NTP synchronisation is used) › syncStatus should be OK – Synchronization=1 › nodeSystemClock should be in LOCKED_MODE. › syncReference should show the correct reference (NTP or GPS) active and configured Enabled Features › User throughput can be limited by the available/installed licenses › The following features directly impact end user throughput – Downlink/Uplink Baseband Capacity – Channel Bandwidth (5, 10, 15 and 20) MHz – 64-QAM DL / 16-QAM UL – Dual Antenna DL Performance Package › To quickly check active licenses (including states): – moshell> inv Licensing (9/1551-LZA 701 6004 ) Expected Throughput (Simplified) DL Scheduling Block (SB) -> Bit calculation (Normal CyclicPrefix) dlCyclicPrefix = 15 KHz => 7 OFDM symbols Resource Elements (RE) per Resource Block (7 OFDM symbols x 12 SubCarriers) RE per SB 2 x RB RS RE (per RB) RS RE (per SB) Control Region Size (CRS) in OFDM symbols nrOfSymbolsPdcch RE per CRS (OFDM*12 - 4 RS Tx) (OFDM*12 - 8 RS MIMO) Tot Num RE per SB available for PDSCH (best case w/o SCH/BCH) Bits per SB - QPSK (2) Bits per SB - 16QAM (4) Bits per SB - 64QAM (6) Tx Diversity 2x2 MIMO 84 168 168 8 16 336 16 32 1 2 3 1 2 3 8 20 32 16 40 64 144 288 576 864 132 264 528 792 120 240 480 720 288 576 1152 1728 264 528 1056 1584 240 480 960 1440 20 MHz => 100 RB (64 QAM) 86.4 79.2 72 172.8 158.4 144 15 MHz => 75 RB (64 QAM) 64.8 59.4 54 129.6 118.8 108 10 MHz => 50 RB (64 QAM) 43.2 39.6 36 86.4 79.2 72 5 MHz => 25 RB (64 QAM) 21.6 19.8 18 43.2 39.6 36 76 152 304 456 64 128 256 384 52 104 208 312 152 304 608 912 128 256 512 768 104 208 416 624 Max Theoretical L1 Thrpt (Mbps) Tot Num RE per SB available for PDSCH (worst case with SCH/BCH in SB) SCH = 24, BCH = 4 x 12 - 4 per CW Bits per SB - (QPSK) Bits per SB - (16QAM) Bits per SB - (64QAM) Identify the domain › Further analysis required: – Our basic checks have come up short – Throughput issues exist that require advanced/additional analysis › Analysis steps to perform: – Single UE call scenario – Send UDP type traffic in DL/UL direction (e.g. Iperf) – Monitor close to or on the RBS (e.g. Wireshark) – Optionally use a radio monitor (e.g. TEMS) › Decide - Radio or Transport analysis: – Radio issues provide more control for LTE RAN analysis – Transport issues blend/carry-on towards core elements Radio Analysis Radio Analysis › From the domain analysis previously, we believe the Radio may be affecting user throughput › We’ve previously ruled out configuration and MO status using the basic checks › The following slides will cover the various components which make up the radio domain and help to pinpoint the source of poor throughput. › We rely on the baseband scheduler traces and signal traces (mtd) between blocks. Radio Analysis › Ericsson’s LTE Baseband provides a detailed mechanism for tracing the complete L1 and L2 interaction, including MAC scheduling decisions and L1 decoding results. › Using this information we can further isolate the cause of the problem and pinpoint either: – UE problem › Cannot detect ACK/NACK? › Invalid UE reports? – Uu air interface problem – eNB problem › Incorrect setting or non-optimal combination of settings › Scheduling abnormality › Limitation in current eNB software – eNB northbound problem › S1 user plane › Application Server › Core network, SASN/SGW, etc RADIO ANALYSIS › To perform targeted radio analysis, it’s useful to know radio aspects specific to the following traffic scenarios: 1. Downlink 2. Uplink 3. Both uplink and downlink › Post-processing tools will be briefly demonstrated Radio Analysis - Downlink › Areas of analysis for Downlink: – CQI (Channel Quality Index) and RI (Rank Indicator) reported from UE. – Transmission Mode: MIMO (tm3) vs. TxD (tm2) vs. SIMO (tm1) – MCS vs. number of assigned PRBs vs. assignable bits in scheduler – UE Scheduling percentage of TTIs (how often is the UE scheduled) – CFI (number of OFDM symbols for PDCCH) vs. MCS vs. % scheduling – HARQ – RLC retransmissions Radio Analysis – Uplink › Areas of analysis for Uplink: – Uplink scheduling overview – BSR (Buffer Status Report) – PHR (Power Headroom Report) – is the UE at maximum power? – Cell bandwidth vs. maximum allowable PRBs – Link Adaptation – MCS available and 16QAM – PDCCH SIB scheduling colliding with UL grant – HARQ (less important, because we can measure SINR) Radio Analysis DL – CQI/RI and TM › The eNB needs knowledge of the SINR conditions of downlink transmission to a UE in order to select the most efficient MCS/PRB combination for a selected UE at any point in time. › Channel Quality Index (CQI): – Is a feedback mechanism from UE to eNB – Informs eNB of current channel conditions as seen at UE – Directly maps to 3GPP defined modulation/code rate (TS36.213 Table 7.2.3-1) › Defined as the highest coding rate the UE could decode at 10% BLER on HARQ rv=0 transmission – CQI 1-6 map to QPSK – CQI 7-9 map to 16QAM – CQI 10-15 map to 64QAM › Rank Indicator (RI) – Is a feedback mechanism from UE to eNB – Informs eNB whether UE can successfully decode RS from 1 or 2 (or more) antennas. – eNB scheduler uses this feedback to transmit with either: Radio analysis DL – CQI/RI and TM CQI PUSCH DL frequency band PUCCH › The UE measures DL channel quality and reports to eNodeB in the form of Channel Quality Information (CQI) › The average CQI (periodic-CQI reporting) for the whole band (wide-band CQI) is reported periodically on PUCCH (or on PUSCH if user data is scheduled in that TTI) with configured periodicity. PUCCH CQI polling › Sub-band CQI (aperiodic-CQI reporting) is reported when requested by the eNB. This report is for the PDSCH. Report sent on PUSCH. – CQI polling is triggered on demand by eNB based on DL traffic activity. › When 2 antennas are configured, Rank Indicator is also reported. Precoding Matrix Indicator (PMI) also reported in case of transmission mode 4 (not in L11A). Radio Analysis DL – CQI/RI and TM › In order to transmit with MIMO (OLSM) we should check the following: – eNB cell is configured with two working transmit antennas. › Check EUtranCellFDD::noOfUsedTxAntennas > 1 › L11A GA (default) system constant SC125:3 means that tm3 is used in case 2 TX antennas are defined. › If only one TX antenna is configured, then tm1 is used › In order to force Transmit Diversity (i.e. prevent OLSM), SC125:2 must be set – UE CQI/RI report from UE shows RI > 1 › Rank 1: TxDiversity (transmission mode 2, tm2) › Rank 2: MIMO (Open Loop Spatial Multiplexing in L11A) (transmission mode 3, tm3) › mtd peek -ta ulMacPeBl -signal LPP_UP_ULMACPE_CI_UL_L1_MEAS2_DL_IND -dir OUTGOING – This signal (from L1 to MAC scheduler) shows the reported CQI and RI – (also shows HARQ ACK/NACK for downlink data transmission) – (also shows rxPowerReport and timingAdvanceError) Radio Analysis DL – CQI/RI and TM mtd peek -ta ulMacPeBl -signal LPP_UP_ULMACPE_CI_UL_L1_MEAS2_DL_IND -dir OUTGOING LPP_UP_ULMACPE_CI_UL_L1_MEAS2_DL_IND UpUlMacPeCiUlL1Meas2DlIndS { cfrPucch { cfrInfo { ri = 0, cfrLength = 4, cfrFormat = 0, cfrValid = 1, cfrExpected = 1, cfrCrcFlag = 1 }, cfr[] = [0, 0] as hex: [00 00 00 00] } cfrFormat=0 is a WCQI report only (ignore RI) Valid report if cfrValid=1,cfrExpected=1,cfrCrcFlag=1 cfrPusch { cfrInfo { ri = 2, cfrLength = 22, cfrFormat = 4, cfrValid = 1, cfrExpected = 1, cfrCrcFlag = 1 }, cfr[] = [61440, 0, 0, 0] as hex: [f0 00 00 00 00 00 00 00] } cfrFormat=4 is a SCQI + RI report Rank Indicator = 2 (indicates UE can decode both antenna streams) WCQI is first half octet (f => 15). Octets thereafter are subband CQI reports for each RBG. A number of subband CQIs follow (see next slide) cfrPusch { cfrInfo { ri = 2, cfrLength = 18, cfrFormat = 4, cfrValid = 1, cfrExpected = 1, cfrCrcFlag = 1 }, cfr[] = [48969, 49152, 0, 0] as hex: [bf 49 c0 00 00 00 00 00] } WCQI = 11. 5MHz bandwidth means 4PRBs subbands. SCQI = F49C = 11 11 01 00 10 01 11 00 SCQI PRBs: 0-3 -1, 4-7 -1, 8-11 +1, 12-15 0, 16-19 +2, 20-23 +1, 24 -1 Radio Analysis DL – SCQI Visualisation › From the previous slide, SCQI is visualised here.. – For 5MHz, each RBG is 4 PRBs wide (except for SCQI group 7) – SCQI is given relative to WCQI which was 11 in this example SCQI PRBs: 0-3 -1, 4-7 -1, 8-11 +1, 12-15 0, 16-19 +2, 20-23 +1, 24 -1 CQI value Sub-band ( 10 1 10 12 11 13 12 10 ) 2 3 4 5 6 7 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 5 MHz f Radio analysis DL – CQI/RI and TM › cfrFormat = 4 consists of: – 4 bit Wideband CQI (i.e. CQI across whole bandwidth) – Up to 13 subband CQI differentials (depends on bandwidth of cell) › Subband CQI (3GPP TS36.211 Ch 7.2.1) – RBG width depends on bandwidth: › 3 & 5MHz – subband width 4 PRBs › 10MHz – subband width 6 PRBs › 15 & 20MHz – subband width 8 PRBs – Subband Differential mapping, see table below: Radio analysis DL – CQI/RI and TM › 7 possible cfrFormats defined in L11A. › Typically see reports cfrFormat 0 and 4 as described previously › Note that PMI is not yet used (requires tm4) cfrFormat Report includes 0 WCQI 1 RI 2 WCQI + WPMI 3 SCQI 4 SCQI + RI 5 WCQI + SPMI + RI 6 SCQI + WPMI + RI Radio analysis DL – CQI/RI and TM › Transmission Mode and MCS can be traced out with the following: lhsh gcpu01024 te e trace4 UpcDlMacCeFt_DL_SCHEDULER ULMA4/UpcDlMacCeFt_DL_SCHEDULER LEVEL2 cellId=12 : Selected SE and HARQ: rnti=61 bbUeRef=201327456 HARQ idx=1 tbs={7992 0} mcs={18 0} noOfSBs={4294443008 0} rv={0 1} ndi={0 0} rmGbits={21600 0}" MCS for each codeword. In this case, tm2 so only one MCS listed. mtd peek -ta dlMacPeBl -signal LPP_UP_DLMACPE_CI_DL_UE_ALLOC_IND -dir INCOMING -filter {(U16SIG)8,NEQ,(U16)0x00} LPP_UP_DLMACPE_CI_DL_UE_ALLOC_IND (330) UpDlMacPeCiDlUeAllocIndS { sfn = 280 subframeNr = 7 l1Control { transmissionMode = 2 prbResourceIndicatorType = 0 prbList[] = [4294443008, 0, 12, 0]dec [ff f8 00 00 00 00 00 00 00 00 00 0c 00 00 00 00]hex commonTb { newDataFlag = 1, tbSizeInBytes = 999, l1Tb { rvIndex = 0, modType = 2 (UPDLMACPEMode64Qam), nrOfRateMatchedBits = 21600, rmSoftBits = 1237248 } } PRB list in RBGs, for 5MHz RBG size is 2. fff8 corresponds to 25 PRBs (last PRB is 1 less). MCS is a combination of tbSize and modType. 999 bytes = 7992 bits then put into TS36.213 Table 7.1.7.2.1-1 for NPRB=25. That gives ITBS of 16. Convert ITBS to MCS using Table 7.1.7.1-1. Radio Analysis DL – CQI/RI and TM › RBG for Resource Allocation Type 0 – Defined in 3GPP TS36.213 Ch 7.1.6.1 – One bit used to represent a certain number of consecutive PRBs – 1.4MHz is RBG size 1 – 3 & 5MHZ is RBG size 2 – 10MHz is RBG size 3 – 15 & 20MHz is RBG size 4 Radio analysis DL – CQI/RI and TM › Example of switching transmission modes based upon RI (bfn:3352, sfn:280, sf:5.47, bf:128) ULMA4/UpcDlMacCeFt_DL_SCHEDULER LEVEL2 cellId=12 : Selected SE and HARQ: rnti=61 bbUeRef=201327456 HARQ idx=1 tbs={7992 0} mcs={18 0} noOfSBs={4294443008 0} rv={0 1} ndi={0 0} rmGbits={21600 0}" TM=2 transmission with MCS 18 cfrPusch { cfrInfo { ri = 2, cfrLength = 18, cfrFormat = 4, cfrValid = 1, cfrExpected = 1, cfrCrcFlag = 1 }, cfr[] = [48969, 49152, 0, 0] as hex: [bf 49 c0 00 00 00 00 00] } Rank Indicator = 2 received from UE. eNB will now switch to tm3 (OLSM MIMO) transmission WCQI 11 + SCQI. bfn:3352, sfn:280, sf:6.47, bf:131) ULMA4/UpcDlMacCeFt_DL_SCHEDULER LEVEL2 cellId=12 : Selected SE and HARQ: rnti=61 bbUeRef=201327456 HARQ idx=0 tbs={5736 5736} mcs={13 13} noOfSBs={4294443008 0} rv={0 0} ndi={0 1} rmGbits={14400 14400}" TM=3 with MCS=13. TBS = 5736 => 11448 over NPRB=50 25 PRBs as according to previous example Radio Analysis DL – Assignable Bits › If UE is sending with high CQI (in the range 10-15) and RI=2 but throughput is still very low, then the next check should be assignable bits. › Assignable bits means the amount of data in the downlink buffer available for the scheduler to schedule for this UE. › A classic symptom of low assignable bits is that the UE is scheduled with a high MCS but a low number of PRBs. – The scheduler always attempts to send with the highest possible MCS and least number of PRBs so that left-over PRBs could be assigned to another UE. › Another symptom is that the UE is not scheduled every TTI (and nothing else is available to schedule). Radio Analysis DL – Assignable Bits › Possible causes for low assignable bits: 1. RLC STATUS messages are not being received fast enough and RLC buffers are full. › Until RLC STATUS ACK messages are received, already transmitted RLC SDUs are kept in memory in UE and/or eNB › Check for RLC DISCARDs but low (or 0) assignable bits 2. Data received from core network is not enough to fill the RLC buffers in eNB. › Check that non-TCP based traffic is not being sent with too large packet size. For iperf based traffic, recommended size 1360 bytes (default is 1470). › Set MTU of 1360 in UE (or UE laptop). › RLC DISCARDs will trigger TCP congestion control and lower thpt. › In L11A the default RLC buffer size per RB is 750 IP packets – Trace discards with lhsh gcpu00768 te e all UpDlPdcpPeFt_DISCARD – Discards on UDP traffic will not affect throughput – Discards on TCP traffic will trigger TCP congestion control (lower thpt.) Radio Analysis DL – Assignable Bits lhsh gcpu00768 te e all UpDlPdcpPeFt_DISCARD ULMA3/UpDlPdcpPeFt_DISCARD TRAFFIC_ABNORMAL Discarding DL PDCP PDU due to exceeding limits. maxBufferedPacketsInRlc=751 totalNumNonAckedDrbPackets=751 cellId=12 bbUeRef=201327456 bbBearerRef=201327458 receiveFromTeid=3779046158 payloadLength=1506 bytes incl GTP-U header. hoState=0" 750 is default PDCP/RLC buffer per UE in eNB (L11A) TRAFFIC_ABNORMAL corresponds to trace1. Traffic discards for UDP are normal, but for TCP traffic it will cause severe throughput degradation lhsh gcpu01024 te e trace4 UpcDlMacCeFt_DL_SCHEDULER ULMA4/UpcDlMacCeFt_DL_SCHEDULER LEVEL3 cellId=12 : Selected SE and PQ: rnti=61 bbUeRef=201327456 PQ lcid=1 assignableBits=0 minPduSize=56 selectedHarq=0" ULMA4/UpcDlMacCeFt_DL_SCHEDULER LEVEL3 cellId=12 : Selected SE and PQ: rnti=61 bbUeRef=201327456 PQ lcid=2 assignableBits=0 minPduSize=56 selectedHarq=0" ULMA4/UpcDlMacCeFt_DL_SCHEDULER LEVEL3 cellId=12 : Selected SE and PQ: rnti=61 bbUeRef=201327456 PQ lcid=3 assignableBits=8554024 minPduSize=56 selectedHarq=0" About 1MByte of data available for scheduling. Check for low value of assignable bits which indicates e2e problems affecting data available to schedule on air for eNB. Low assignable bits for UDP traffic may indicate MTU problems. LCID 3 is for the default bearer. LCID 1 and 2 for SRB Radio Analysis DL – CFI and Scheduling › Another cause of low (or lower than expected) throughput is that the UE is not being scheduled in every TTI. › This may be caused by: – Limitations in current scheduler implementation – 3GPP defined compromises between control channel efficiency and scheduling efficiency (especially for lower number of users) › L11A software has some limitations to be aware of: – Only one SE per TTI is supported in L11A – SIBs are scheduled in the same was a user data (i.e. they are sent to the scheduler). – When a SIB is transmitted, no user data can be transmitted in the DL at the same time (using default parameters). › It is possible to use (System Constant) SC43 to enable 2SE/TTI in DL › It is possible (only for demo use) to temporarily disable SIB scheduling Radio Analysis DL – CFI and Scheduling › SIBs require PDCCH resources › Typically SIBs consume 4 or 8 CCEs of PDCCH resources. › If a UE is in good SINR conditions, the scheduler may allocate only one CCE for that UE. – In that case, because of limited positions in PDCCH, it is quite likely that a PDCCH collision occurs (especially in low system bandwidths) › If a UE is in bad SINR conditions, the scheduler may allocate a large number of CCEs for that UE (2 or 4 or 8 CCEs) – Depending on the configured CFI there may only be common search space available or it may still collide with other PDCCH users. › See Radio Analysis UL – PDCCH slides for more details Radio Analysis Dl – HARQ › Each transport block transmission is represented as a HARQ process. – Each HARQ process data is held in memory until NDI is toggled (i.e. New data is to be sent). – This allows fast retransmission of erronerously received data. › The schedulers representation of an HARQ process is as follows: – Feedback status › (ACK, NAK, DTX, PENDING) – TBS – transport block size – MCS – modulation and coding scheme – RV – redundancy version. HARQ has 4 redundancy versions, rv0, rv2, rv3, rv1. – NDI – New Data Indicator (physical layer bit toggled for new data). › Do not confuse with newDataFlag which is scheduler internal flag where 1 means new data and 0 means retransmission. – Number of transmission attempts (max 4 transmissions in L11A default paramters) › In case of rank 2 spatial multiplexing there are 16 HARQ process per UE instead of 8, but there are two processes that share the same ID – Scheduler sees them as separate processes that are coupled to each other Radio Analysis Dl – HARQ Example › The following slides will show an example of tracing out downlink HARQ – Initial downlink grant is sent with rv=0 (MIMO, 2 codewords) › SFN 280/subframe 8 – HARQ NACK received on both code words › SFN 281/subframe 2 (DL Grant + 4TTI) – First retransmission sent with rv=2 › SFN 281/subframe 6 (8 TTI past initial transmission is earliest occasion) – HARQ ACK received on both code words › SFN 282/subframe 0 (DL Grant ReTx + 4TTI) Radio Analysis Dl – HARQ DL Grant LPP_UP_DLMACPE_CI_DL_UE_ALLOC_IND (330) UpDlMacPeCiDlUeAllocIndS { sfn = 280 SFN/subframe where DL PDSCH will occur. subframeNr = 8 PDCCH DL Grant sent at same sfn/subframe. ueAlloc[0] { RNTI, TM, used PRBs (same for both code words) l1Control { rnti = 61 transmissionMode = 3 prbResourceIndicatorType = 0 prbList[] = [4294443008, 0, 12, 0]dec [ff f8 00 00 00 00 00 00 00 00 00 0c 00 00 00 00]hex swapFlag = 0 If re-transmission, this indicates if CW0 and CW1 swapped layers } newDataFlag indicates if it is new data or not nrOfTb = 2 tbAlloc[0] { HARQ redundancy version. rv0 used for initial transmission, rv2, tbIndex = 0 rv3, rv1 used for re-transmission. commonTb { newDataFlag = 1, tbSizeInBytes = 717, l1Tb { rvIndex = 0, modType = 1 (UPDLMACPEMode16Qam), nrOfRateMatchedBits = 14400, rmSoftBits = 1237248 } } macTb { dlHarqProcessId = 0, nrOfMacCtrlElem = 0 } rlcTb { nrOfBearer = 1, bearerAlloc[0] { bbBearerRef = 201327458, lcid = 3, rbScheduledSizeInBytes = 717 } } } HARQ process number. 8 HARQ processes tbAlloc[1] { exist in FDD LTE L11A. ... CW1 defined here. Radio Analysis Dl – HARQ FEEDBACK (NACK/NACK) LPP_UP_ULMACPE_CI_UL_L1_MEAS2_DL_IND (431) UpUlMacPeCiUlL1Meas2DlIndS { sigNo = 23070220 header { SFN/subframe +4 from DL grant (i.e. where the cellId = 12 HARQ ACK/NACK is received from UE). sfn = 281 subFrameNo = 2 HARQ NACK received for DL HARQ Process 0 on both code } words. nrOfPuschReports = 0 nrOfPucchReports = 1 DetectedHarqIndication: 0 => NACK/NACK, 1 => NACK/ACK, totalNrOfReports = 1 2 => ACK/NACK, 3 => ACK/ACK, 4 => DTX (nothing received) reportList[0] { pucchReport { meas2DlUlReportType = 1 (ElibBbBaseCommonMeas2DlPucchReport) bbUeRef = 201327456 isDtx { isDtx = 0 } dlHarqInfo { dlHarqValid = 1, detectedHarqIndication = 0, dlHarqProcessId = 0, nrOfTb = 2, swapFlag = 0 } rxPower { prbListStart = 0, prbListEnd = 0, rxPowerReport = -1150, sinr = 0 } timingAdvanceError { timingAdvanceError = 1 } cfrPucch { cfrInfo { ri = 0, cfrLength = 0, cfrFormat = 0, cfrValid = 0, cfrExpected = 0, cfrCrcFlag = 0 }, cfr[] = [0, 0] as hex: [00 00 00 00] } } } } Radio Analysis Dl – HARQ ReTX LPP_UP_DLMACPE_CI_DL_UE_ALLOC_IND (330) UpDlMacPeCiDlUeAllocIndS { sfn = 281 SFN/subframe where DL PDSCH will occur. subframeNr = 6 PDCCH DL Grant sent at same sfn/subframe. ueAlloc[0] { RNTI, TM, used PRBs (same for both code words) l1Control { rnti = 61 Same as previous transmission transmissionMode = 3 prbResourceIndicatorType = 0 prbList[] = [4294443008, 0, 12, 0]dec [ff f8 00 00 00 00 00 00 00 00 00 0c 00 00 00 00]hex swapFlag = 0 newDataFlag=0 means it’s a retransmission } nrOfTb = 2 HARQ redundancy version. rv2 is used for first retransmission tbAlloc[0] { tbIndex = 0 commonTb { newDataFlag = 0, tbSizeInBytes = 717, l1Tb { rvIndex = 2, modType = 1 (UPDLMACPEMode16Qam), nrOfRateMatchedBits = 14400, rmSoftBits = 1237248 } } macTb { dlHarqProcessId = 0, nrOfMacCtrlElem = 0 } rlcTb { nrOfBearer = 0 } HARQ process number (same as before) } tbAlloc[1] { CW1 defined here. tbIndex = 1 commonTb { newDataFlag = 0, tbSizeInBytes = 717, l1Tb { rvIndex = 2, modType = 1 (UPDLMACPEMode16Qam), nrOfRateMatchedBits = 14400, rmSoftBits = 1237248 } } Radio Analysis Dl – HARQ FEEDBACK (ACK/ACK) LPP_UP_ULMACPE_CI_UL_L1_MEAS2_DL_IND (431) UpUlMacPeCiUlL1Meas2DlIndS { sigNo = 23070220 header { SFN/subframe +4 from DL grant (i.e. where the cellId = 12 HARQ ACK/NACK is received from UE). sfn = 282 subFrameNo = 0 HARQ ACK/ACK received for DL HARQ Process 0 on both } code words. nrOfPuschReports = 0 nrOfPucchReports = 1 DetectedHarqIndication: 0 => NACK/NACK, 1 => NACK/ACK, totalNrOfReports = 1 2 => ACK/NACK, 3 => ACK/ACK, 4 => DTX (nothing received) reportList[0] { pucchReport { meas2DlUlReportType = 1 (ElibBbBaseCommonMeas2DlPucchReport) bbUeRef = 201327456 isDtx { isDtx = 0 } dlHarqInfo { dlHarqValid = 1, detectedHarqIndication = 3, dlHarqProcessId = 0, nrOfTb = 2, swapFlag = 0 } rxPower { prbListStart = 0, prbListEnd = 0, rxPowerReport = -1152, sinr = 0 } timingAdvanceError { timingAdvanceError = 0 } cfrPucch { cfrInfo { ri = 0, cfrLength = 0, cfrFormat = 0, cfrValid = 0, cfrExpected = 0, cfrCrcFlag = 0 }, cfr[] = [0, 0] as hex: [00 00 00 00] } } } } Radio Analysis DL – RLC › RLC retransmissions are triggered: 1. When HARQ fails to transmit a transport block within the maximum number of configured retransmissions – Default number of HARQ transmissions is 4 in L11A 2. If RLC STATUS messages are not received within the time frames configured › RLC STATUS messages are sent between peer nodes (eNB and UE) to inform about lost RLC packets. They can be traced out using – › mtd peek -ta dlRlcPeBl -si UP_DLRLCPE_FI_STATUS_FOR_DL_TRAFFIC_IND Check: – – – ACK_SN should be increasing, otherwise RLC buffers are not released NACK_SN indicates RLC retransmissions (occasionally is OK) DataRadioBearer::tStatusProhibit governs how often RLC STATUS messages may be generated, default is 25ms in L11A. › A too low value will produce too many RLC control messages › A too high value may cause RLC buffers to become exhausted Radio Analysis DL – RLC 0xd4205d4f=(sfn:322, sf:0.33, bf:212): UP_DLRLCPE_FI_STATUS_FOR_DL_TRAFFIC_IND (343) UpDlRlcPeRlcStatusForDlTrafficIndS { RLC PDU { Indicates the SN (Sequence Number) of the last D/C = 0 (Control PDU) successfully received RLC packet CPT = 000 (STATUS PDU) ACK_SN = 743 E1 = 0 (A set of NACK_SN, E1 and E2 does not follow) } NACK_SN indicates RLC retransmissions (HARQ failures) } 0xd4205d4f=(sfn:324, sf:5.33, bf:212): UP_DLRLCPE_FI_STATUS_FOR_DL_TRAFFIC_IND (343) UpDlRlcPeRlcStatusForDlTrafficIndS { RLC PDU { Next RLC STATUS received 25ms later, check D/C = 0 (Control PDU) that it’s coming regularly CPT = 000 (STATUS PDU) DataRadioBearer::tStatusProhibit=25ms default ACK_SN = 787 E1 = 0 (A set of NACK_SN, E1 and E2 does not follow) } Check that ACK_SN is increasing or RLC buffers not released } Radio Analysis – Uplink › Areas of analysis for Uplink: – Uplink scheduling overview – BSR (Buffer Status Report) – PHR (Power Headroom Report) – is the UE at maximum power? – Cell bandwidth vs. maximum allowable PRBs – Link Adaptation – MCS available and 16QAM – PDCCH SIB scheduling colliding with UL grant – HARQ (less important, because we can measure SINR) Radio Analysis UL – UPlink Scheduling › Scheduling request, SR (PUCCH) UE requests UL resources › UL Grant (PDCCH) Scheduler assigns initial resources Channel state info UL UL scheduler › Buffer status report (PUSCH) transmitted in UL › UL grant (PDCCH) transmitted (valid per UE) › Data is transmitted (PUSCH) eNodeB › Channel sounding Ue Radio Analysis Ul – BSR › Buffer Status Report (BSR) is used to inform the eNB of the current data waiting for transmission in the UE (3GPP TS36.213 Ch. 6.1.3.1) › Values ranges from 0 up to >15000 bytes using 64 index values. – e.g. index 0 for BS=0, index 1 for 0 < BS <= 10 and so forth. › Can be traced out through LPP_UP_ULMACPE_CI_UL_MAC_CTRL_INFO_IND. Expect to see high values for maximum UL throughput. Low values indicate UE/laptop problem. Type of MAC report, this case short BSR (6) macCtrlElementList[0] { type = 6 (UpUpCommonMacCommonMacCtrlElemShortBsr) powerHeadroomReport { type = 6 (UpUpCommonMacCommonMacCtrlElemShortBsr), powerHeadroom = 127 } cRnti { type = 6 (UpUpCommonMacCommonMacCtrlElemShortBsr), crnti = 8362387 } truncatedBSR { type = 6 (UpUpCommonMacCommonMacCtrlElemShortBsr), bufferSize = 127 } shortBSR { type = 6 (UpUpCommonMacCommonMacCtrlElemShortBsr), bufferSize = 127 } longBSR { type = 6 (UpUpCommonMacCommonMacCtrlElemShortBsr), bufferSizeNr1Nr2 = 127, bufferSizeNr3Nr4 = 39315 } } LSB 6 bits are the BSR index (this case >150000 bytes) MSB 2 bits is the LCID Radio Analysis UL – PHR › Power Headroom Report (PHR) is used to inform the eNB of the remaining transmit power available at the UE. (3GPP TS36.321 Ch. 6.1.3.6) › Defined as difference between configured maximum UE output power and estimated power used for PUSCH transmission › Reports a index value similar to BSR with values between -23 up to 40 dB › PH values are close to (or less than) 0 means the UE is power limited – Ideally we look for positive values somewhat greater than 0 – When a UE is power limited, the eNB may schedule fewer PRBs in order to reduce the required output power of the UE, this can in turn reduce throughput. Radio Analysis UL – PHR Type of MAC report, this case PHR (3) macCtrlElementList[0] { type = 3 (UpUpCommonMacCommonMacCtrlElemPowerHr) powerHeadroomReport { type = 3 (UpUpCommonMacCommonMacCtrlElemPowerHr), powerHeadroom = 55 } cRnti { type = 3 (UpUpCommonMacCommonMacCtrlElemPowerHr), crnti = 3637377 } truncatedBSR { type = 3 (UpUpCommonMacCommonMacCtrlElemPowerHr), bufferSize = 55 } shortBSR { type = 3 (UpUpCommonMacCommonMacCtrlElemPowerHr), bufferSize = 55 } longBSR { type = 3 (UpUpCommonMacCommonMacCtrlElemPowerHr), bufferSizeNr1Nr2 = 55, bufferSizeNr3Nr4 = 32897 } } PHR value of 55 which corresponds to 32 <= PH < 33. In this case there is no power limitation on the UE side. PH Index values <= 23 indicates the UE has reached maximum transmission power Negative values indicate the UE was power limited See 3GPP TS36.133 Ch 9.1.8.4 for index mapping Radio Analysis UL – PUCCH and PUSCH › PUCCH takes a minimum 1 PRB on each side of the uplink band for uplink control signalling, reducing the size of PUSCH – E.g. 5MHz bandwidth, 25 PRBs available. Minimum 2 PRBs for PUCCH. – 23 PRBs available for PUSCH Radio Frame 0 1 2 3 4 5 PUCCH Cell Bandwidth PUSCH PUCCH 6 7 8 9 Time (ms) PUCCH – Semi-static allocation of CQI, SR, ACK/NAK PUSCH – Used for UE data scheduling and UL RA msgs PUCCH – Semi-static allocation of CQI, SR, ACK/NAK Radio Analysis UL – PRB Limitations › Due to 3GPP specified design limitations in the UL it is not always possible to utilise all free PRBs for UL transmissions › 3GPP TS36.211 Ch 5.3.3 defines the following formula for the number of PRBs on PUSCH for a single transmission: 2 3 5 – Where a, b and c are integers. – For 5MHz: › 23 PRBs are available for PUSCH (2 allocated to PUCCH) › Max number of PRBs for a single PUSCH transmission is 20 PRBs. › This corresponds to a=2, b=0 and c=1 (i.e. 3 PRBs are unavailable to be used). › In L11A, 3 PRBs would be unused (only one SE/TTI possible). › In later releases, 3 PRBs could be used by a second UE. a b c Radio Analysis UL – Link Adaptation Inputs to Uplink Link Adaptation are: ulCellCe cellStatusReportInd(interferencePower (-125 .. -80dB) run every subframe ulMacPe ulL1Meas2UlInd(rxPower(-140 .. 0 dB) UL interference power: LPP_UP_ULCELLPE_CI_CELL_STATUS_REPORT_IND outgoing from ulCellCeBl run every subframe UE transmitts ulMacPe ulL1MacCtrlInfoInd(powerHeadroom(0 .. 63dB)) Received power of UE (across traffical PUSCH PRBs): run every periodicPhrReport LPP_UP_ULMACPE_CI_UL_L1_MEAS2_UL_IND outgoing from ulMacPeBl ulMacPe PHR reports & HARQ CRC (BLER): LPP_UP_ULMACPE_CI_UL_MAC_CTRL_INFO_IND outgoing from ulMacPeBl ulL1MacCtrlInfoInd(CRC) run every subframe UE transmitts Input to Link Adaptation Goal: Select MCS for a certain allocation size to maintain the target BLER (10%) for the first transmission Radio Analysis UL – Link Adaptation LPP_UP_ULCELLPE_CI_CELL_STATUS_REPORT_IND UpUlCellPeCiCellStatusReportIndS { sfn = 456 Cell interference level x 10 (i.e. -117.0 dBm) subFrameNo = 3 interferencePower = -1170 High values here (>-104) LPP_UP_ULMACPE_CI_UL_L1_MEAS2_UL_IND (432) UpUlMacPeCiUlL1Meas2UlIndS { sfn = 264 subFrameNo = 8 nrOfPuschReports = 1 rxPower { prbListStart=1, prbListEnd=48, rxPowerReport=-956, sinr=821854514 } rxPwr = -95.6dBm over those PRBs (pZeroNominalPusch= -96dBm) sinr[dB] = 10 log 10 (sinr 2-22 ) = ~22.9dB LPP_UP_ULMACPE_CI_UL_MAC_CTRL_INFO_IND (433) UpUlMacPeCiUlMacCtrlInfoIndS { sfn = 264 HARQ ACK / PHR subFrameNo = 8 harqInfo = 1 (UpUpCommonMacCommonMacCtrlElemHarqFeedbackAck) macCtrlElementList[0] { type = 3 (UpUpCommonMacCommonMacCtrlElemPowerHr) powerHeadroomReport { type = 3 (UpUpCommonMacCommonMacCtrlElemPowerHr), powerHeadroom = 55 } Radio Analysis UL – Link Adaptation › L11A supports up to MCS 24 in the uplink by default – MCS21-24 are defined as 64QAM – However, according to 3GPP TS36.213 Ch 8.6.1 if a UE does not support 64QAM then 16QAM can be used for MCS21-24. – Check that MCS24 is selected. If not, check link adaptation inputs for problems › In UL, the eNB itself can directly measure SINR of the received signal – Therefore CQI is not necessary for UL transmission – eNB can use Received Power/SINR, PHR, UL interference and UL HARQ BLER measurements to control MCS Radio Analysis UL – Link Adaptation › Check for: – High values of UL interference › Could there be some external interferer? › Are the values of pZeroNominalPusch in neighbour cells too high? – rxPower too low › Target is EUtranCellFDD::pZeroNominalPusch. Is it set too high? – PHR shows UE at maximum Tx power › Is EUtranCellFDD::pZeroNominalPusch too high causing UE to exceed maximum transmit power? › Closed-loop power control TPC ignored by UE? – Low values of SINR › Is EUtranCellFDD::pZeroNominalPusch too low? › Closed-loop power control TPC ignored by UE? Radio Analysis UL – PDCCH Radio Frame 0 1 2 In case many PDCCH CCEs are used for DL transmission (e.g. SIB with 8 CCEs) it may be that UL grant is not possible to be scheduled in this TTI for a single UE! Note: PCFICH and PHICH multiplex into the DL subframe red area marked PDCCH PDCCH PDCCH carries both the UL (PUSCH) assignment and DL (PDSCH) assignment. 3 4 5 6 7 8 9 Time (ms) PDSCH DL subframe (current) PUSCH UL subframe (4 TTI later) Radio Analysis UL – PDCCH › PDCCH is used to signal: – Downlink (PDSCH) assignments – Uplink (PUSCH) grants › In case of a downlink SIB transmission, 8 CCEs of PDCCH may be used for downlink grant. › To reduce processing load when decoding PDCCH, 3GPP defines particular search spaces within PDCCH depending on: – Number of CCEs for grant – Number of CCEs for PDCCH – RNTI of the UE › Depending on these parameters, it may not be possible to allocate a PDCCH uplink grant resource and therefore the UE may not be able to be scheduled every TTI even if there are unused PUSCH resources. See 3GPP TS36.213 Ch 9.1.1 See KO link: PDCCH visualisation KO Radio Analysis UL – PDCCH Search space for 1 CCE completely overlaps 8 CCE search space. In this example, DL SIB transmission completely prevents any UL grant for this UE RNTI 516 in subframe 5 From KO link: PDCCH visualisation KO Radio Analysis UL – HARQ › LTE defines uplink with synchronous HARQ to reduce PDCCH signaling load and simplify the uplink HARQ processing › Example 1, successfully received PUSCH data: – Subframe n: UL grant sent to UE – Subframe n+4: PUSCH data received (rv=0) – Subframe n+8: ACK sent, UL grant with New Data Indicator toggled – Subframe n+12: new PUSCH data received (new HARQ process) › Example 2, HARQ retx: – Subframe n: UL grant sent to UE – Subframe n+4: PUSCH data received (rv=0) – Subframe n+8: NACK sent, NO UL grant is signaled on PDCCH – Subframe n+12: PUSCH data received (rv=2) (same HARQ process) – Subframe n+16: ACK/NACK, etc up to max number of retx Radio Analysis UL – HARQ ReTx PUCCH PUSCH for scheduling SE1 n SE2 PUCCH SE3 Assume all three non-correctly decoded (CRC not ok) (N)ACK Colliding ACK n+8 PRACH Postponed reTx time NACK Grant needed Non-Adaptive reTx Adaptive reTx Radio Analysis UL – HARQ › Because of the synchronous nature of Uplink HARQ, the following scheduling priority is used: – Random Access Message 3 (RRC Connection Request). Scheduled 6 subframes before, special case. – Non-adaptive HARQ retransmission – Adaptive HARQ retransmission – New Data transmission › Non-adaptive means no UL grant is explicitly scheduled for the retransmission › Adaptive means that scheduling collision occurred (e.g. collision with PRACH) and an explicit UL grant was signalled to: – Move the allocated PRBs to another part of the UL spectrum – Suspend (delay) the UL HARQ retransmission for 8 TTIs later Radio Analysis – Traffic Abnormal › TRACE1 in baseband is defined as TRAFFIC_ABNORMAL. It should be used to trace out abnormal conditions in baseband processing. – Normally the output gives a good description of the problem encountered › Some useful TRAFFIC_ABNORMAL traces: – – – – – – – lhsh gcpu01024 te e trace1 UpcDlMacCeBl lhsh gcpu01024 te e trace1 Upc* lhsh gcpu00256 te e trace1 UpUlMacPeBl_Smac lhsh gcpu00768 te e trace1 ElibPapBlEth lhsh gcpu00256 te e trace1 UpUlMacPeBl_Smac lhsh gcpu00768 te e trace1 UpDlL1PeFt_DEADLINE_MISSED lh gcpu fte e trace1 .*_DISCARD Radio Analysis – Post-Processing Tools › 3GPP has specified L1 messages in order to reduce the bits required for transmission on the air interface. – These formats can be difficult to read › For this reason, many values in the traces are presented in formats which require conversion to human readable formats, for example: – PRBs allocated in DL/UL grant messages – PHR values – BSR values – SINR – MIMO HARQ feedback, etc.. › Tools exist to perform these conversions and compact the data presentation to the end user – One such tool is bbfilter or scheduling_filter.pl – Check the flowfox web page for details Radio Analysis – bbfilter Downlink $ cat decoded_dl_log.log | ./bbfilterv2.2 -bw 5 –dl sfn|sf|mode|dlModul|mcs1|mcs2|prb|Ndf|Tbs1|Tbs2|AssBits|Harq|dlBler|cqi|ri| 280| 4|TxDi| 64QAM | 16 | 0 |25 | Y|7736| 0|8771784| | | 11| 2| 280| 5| | | | | | | | | |A | 0% | | | 280| 6|TxDi| 64QAM | 18 | 0 |25 | Y|7992| 0|8764088|A | 0% | | | 280| 7|TxDi| 64QAM | 18 | 0 |25 | Y|7992| 0|8756144|A | 0% | | | 280| 8|Mimo| 16QAM | 13 | 13 |25 |Y Y|5736|5736|8748192|A | 0% | | | 280| 9|Mimo| 16QAM | 13 | 13 |25 |Y Y|5736|5736|8736760| | | | | 281| 0|Mimo| 16QAM | 12 | 12 |25 |Y Y|4968|4968|8737384|A | 0% | | | 281| 1|Mimo| 16QAM | 13 | 13 |25 |Y Y|5736|5736|8763568|N | 0% | | | 281| 2|Mimo| 16QAM | 13 | 13 |25 |Y Y|5736|5736|8776208|N N | 2% | | | 281| 3|Mimo| 16QAM | 13 | 13 |25 |Y Y|5736|5736|8800856|N N | 4% | | | 281| 4|Mimo| 16QAM | 13 | 13 |25 |Y Y|5736|5736|8825504|N N | 6% | | | 281| 5|TxDi| 16QAM | 30 | 0 |25 | N|7992| 0|8862160|N N | 8% | | | 281| 6|Mimo| 16QAM | 30 | 30 |25 |N N|5736|5736|8862200|N N | 10% | | | 281| 7|Mimo| 16QAM | 30 | 30 |25 |N N|5736|5736|8862200|A A | 10% | | | HARQ ACK/NACK refers to the transmission 4 subframes earlier! NOTE: Format modified to fit on slide, only example! Radio Analysis – bbfilter Uplink $ cat decoded_ul_log.log | ./bbfilterv2.2 -bw 5 –ul sfn|sf|rxPwrPus|prb|ulTpc|sinr|ulModul|mcs|ndf|ul bsr 266| 6| -95.6 | 48| 0:1 | 22 | 16QAM | 23| Y | 266| 7| -95.6 | 48| 0:1 | 22 | 16QAM | 24| Y | 266| 8| -95.6 | 48| 0:1 | 22 | 16QAM | 24| N | 266| 9| -95.6 | 48| 0:1 | 23 | 16QAM | 24| Y | 267| 0| -95.7 | 48| 0:1 | 22 | 16QAM | 24| Y |>150000 267| 1| -95.8 | 40| 0:1 | 22 | 16QAM | 24| Y | 267| 2| -95.6 | 48| | 23 | 16QAM | 24| Y | 267| 3| -95.6 | 48| 0:1 | 22 | 16QAM | 24| Y | 267| 4| -95.6 | 48| 0:1 | 22 | 16QAM | 23| Y | 267| 5| -95.6 | 48| 0:1 | 22 | 16QAM | 23| Y |>150000 267| 6| -95.6 | 48| 0:1 | 23 | 16QAM | 24| N |>150000 267| 7| -95.6 | 48| 0:1 | 23 | 16QAM | 24| Y | 267| 8| -95.6 | 48| 0:1 | 22 | 16QAM | 24| Y | |phr | | | | | | | | | | | | | 32 |ul tbs| ul crc |har|ulBler| | 25456| | A | 2% | | 25456| | A | 2% | | 25456| ERR 3182| N | 5% | | 24496| | A | 5% | | 24496| | A | 5% | | 21384| | A | 5% | | 25456| | A | 5% | | 25456| | A | 4% | | 25456| | A | 4% | | 25456| | A | 4% | | 25456| | A | 4% | | 25456| | A | 4% | | 24496| | A | 4% | UL BSR and PHR values decoded NOTE: Format modified to fit on slide, only example! Radio Analysis - Summary › Areas of analysis for Downlink: – CQI / RI (Rank Indicator) reported from UE. – Transmission Mode (MIMO, TxD, SIMO) – MCS vs. number of assigned PRBs vs. assignable bits in scheduler – UE Scheduling percentage of TTIs (how often is the UE scheduled) – PDCCH CFI and scheduling impacts – HARQ – RLC retransmissions › Areas of analysis for Uplink: – BSR (Buffer Status Report) – PHR (Power Headroom Report) – is the UE at maximum power? – Cell bandwidth vs. maximum allowable PRBs – Link Adaptation – MCS available and 16QAM – PDCCH SIB scheduling colliding with UL grant – HARQ (less important, because we can measure SINR) Transport Analysis Transport Analysis › End user data is transferred over the S1-U interface › Several Transport Network topologies (L2/L3) provide great flexibility in design › Several router redundancy methods are supported › Transport network dimensioning provides insights into the peak provisioning on the S1 link › The LTE RBS is a QoS enabler, providing end user and transport network QoS differentiation › Performance management counters and tracing provide us with powerful node observability methods Transport Topology Transport Topology › No strict requirements on using a L2 switched or L3 routed LTE RAN transport network › No specified topology requirement › A router is required in the network, but LTE RAN transport network does not have to be L3 › Network design is important (number of hops for L3 vs. size of broadcast domain for L2) › This topology flexibility could complicate troubleshooting efforts depending on the nodes involved (say 3PP support is required) Transport configuration › A 2 VLAN configuration is recommended (separating O&M and Transport): ComInf Firewall OAM SD O&M GW If O&M GW If SoIP Servers distributed in the network. SoIP Server CN Router/FW MME Pool S-GW Pool S1 GW If S1 GW If Switched Ethernet (Carrier Ethernet) RBS 1 O&M IP CP, UP & SOIP IP RBS n O&M VLAN TN VLAN O&M IP CP, UP & SOIP IP Router Path Supervision (RPS) › RPS provides router redundancy for S1/X2 traffic › RPS is configurable via the IpInterface MO virtual router redundancy protocol (VRRP) › LTE RBS supports VRRP (a router redundancy protocol) › VRRP uses an election method to assign responsibility for a virtual router to one of the VRRP routers on a LAN › The Master VRRP router controls the IP address(es) associated with a virtual router and forwards packets sent to these IP addresses › If the Master fails, one backup VRRP router will act as the virtual router › LTE RBS is transparent to the process, it does not directly participate in VRRP eNB eNB eNB 10.1.1.34 10.1.1.34 Master 10.1.1.35 Backup Transport Dimensioning › Dimensioning of the northbound transport network will impact achievable end user throughput rate › LTE RBS transport network dimensioning process (mobile backhaul): Determine bandwidth needed for last mile Determine cell thpt in a loaded network and avg. cell thpt during busy hour › Dimensioning is based on payload only! Calculate agg. bandwidth required in mobile backhaul Dimensioning methods Method Description Overbooking Allows more users than the dimensioned quantity, using the rationale that only a subset of users is allocating bandwidth at the same time. Overdimensioning Calculates the dimensioned bandwidth required by multiplying the average requirement by an overdimensioning factor. Peak allocation Uses the maximum throughput capacity as the dimensioned link capacity. The link is dimensioned for the maximum possible bit rate. Overprovisioning Monitors the link use. When a predefined use limit is reached on the link, a capacity upgrade is initiated. A general rule is that the use limit is set to 50%. Transport Aggregation BH displacement factor S-GW/ S-GW/ PDN GW PDN GW Dimension for: ΣA2 × 0.8 A3 Dimension for ‘Average eNodeB throughput during Busy Hour’ = 50 Mbit/s per eNB Dimension for ‘eNodeB throughput in a loaded network for a 3x1 configuration’ = 100 Mbit/s per eNB A1 Dimension for peak rate to 1 cell= 150 Mbit/s Input (assumptions) Cell Peak Rate A2 A2 A1 20 MHz Cell 150 Mbps Cell Throughput in a Loaded Network 35 Mbps Peak load for 3x1 in a Loaded Network ~100 Mbps R B S R B S R B S A1 R B S R B S R B S R B S R B S R B S DUL TN Capabilities (L11) › The DUL is equipped with two Gigabit Ethernet physical interfaces – TN-A: Electrical Ethernet (100/1000 Mbps) – TN-B: Electrical/Optical Ethernet (1000 Mbps) › Only 1 interface is activated through configuration (either TN-A or TN-B) › Maximum throughput of 173 Mbps (or 150 Mbps of User Data assuming other overheads such as signalling) › Up to 500 connected users › Note: – Wire tracing is not possible on the DUL – You must use an external product to mirror traffic leaving the DUL Quality of service (qos) › Transport network QoS is a part of the complete LTE QoS concept › QoS, in an IP transport network, is the ability to treat packets/frames differently based on their content › Without QoS, each packet/frame is given equal access to the network resources QOS for user plane Bearers Dedicated Bearer (QoS via PCRF) IP Address Service 1 (e.g. Internet) Default Bearer (QoS via MME) Service 2 (e.g. P2P File Sharing) Service 3 (e.g. IMS-Voice or MTV) Terminal RAN Transport Gateway (Bearer Policy Enforcer) Service Data Flow (SDF) Further Reading: 3GPP TS 23.401 Quality Class Indicators › A Quality Class Indicator (QCI) is used to signal the QoS requirements of a bearer › 23.401 defines 9 standard QCIs, each one with specific characteristics › Operators may define proprietary QCIs to introduce new services Priority Packet Delay Budget Packet Loss Rate 1 2 100 ms 10-2 Conversational Voice 2 4 150 ms 10-3 Conversational Video (Live Streaming) 3 50 ms 10-3 Real Time Gaming 4 5 300 ms 10-6 Non-Conversational Video (Buffered 5 1 100 ms 10-6 IMS Signaling 6 6 300 ms 10-6 - Video (Buffered Streaming) - TCP-based (e.g., www, e-mail, chat, ftp, p2p file sharing, progressive video, etc.) 7 100 ms 10-3 - Voice, - Video (Live Streaming) - Interactive Gaming 300 ms 10-6 - Video (Buffered Streaming) - TCP-based (e.g., www, e-mail, chat, ftp, p2p file sharing, progressive video, etc.) QCI 3 7 8 9 Resource Type GBR Non-GBR 8 9 Example Services Streaming) Transport Network QoS – Layer 3 › QCIs are mapped to IP layer Differentiated Services Code Points (DSCP) IPx is Interface between node and IP transport network Site Infrastructure, IP-backbone and RAN Transport Application IP Transport Network IP IP and Ethernet PPPoE Ethernet Ethernet Version 4 Header Length 4 DiffServ 7 Total Length 16 20 bytes Further Reading: RFC 2474 Differentiated Services Code Point (DSCP) EPC Transport Network QoS – Layer 2 › Use of P-Bits allows prioritizing different types of traffic › Priority queuing, enabling some Ethernet frames to be forwarded ahead of others within a switched Ethernet network › Frames can be assigned to different scheduler queues › Maintain the same value throughout the IP Network to deliver the same QoS IPx is Interface between node and IP transport network Site Infrastructure, IP-backbone and RAN Transport Application IP Transport Network IP IP and Ethernet PPPoE Ethernet Ethernet Preamble 7 SFD 1 DA 6 SA 6 User Priority 3bits Priority Bit (P-bit) TPI 2 TAG 2 CFI 1bit Type 2 VLAN ID 12bits Data 46 to 1500 CRC 4 Further Reading: IEEE 802.1p EPC QOS Mapping › LTE RAN Quality of Service (QoS) ensures the LTE bearer and transport level service requirements QoS Profile QCI IP Header DSCP DATA IP Packet AMBR ARP Mapping: Takes place in RBS / EPC P-bit Mapping: Edge devices handling L2/L3 payload Ethernet Header DATA Ethernet Frame QoS Mapping LTE RAN QoS Configuration › LTE RAN complies with 3GPP TS 23.203 and IEEE 802.1p: =============================================================== ==================================================== MO MO dscp priority qci Attribute Value =============================================================== ==================================================== QciTable=default,QciProfilePredefined=qci1 46 2 1 Subrack=1,Slot=1,PlugInUnit=1,ExchangeTerminalIp=1,Gig aBitEthernet=1 dscpPbitMap t[64] = QciTable=default,QciProfilePredefined=qci7 20 7 7 >>> Struct[0] QciTable=default,QciProfilePredefined=qci9 12 9 9 >>> 1.dscp = 0 QciTable=default,QciProfilePredefined=qci5 40 1 5 >>> 2.pbit = 0 QciTable=default,QciProfilePredefined=qci3 34 3 3 ... truncated ... QciTable=default,QciProfilePredefined=default 0 10 0 >>> Struct[46] QciTable=default,QciProfilePredefined=qci8 10 8 8 QciTable=default,QciProfilePredefined=qci6 28 6 6 QciTable=default,QciProfilePredefined=qci2 36 4 2 QciTable=default,QciProfilePredefined=qci4 26 5 4 has 2 members: has 2 members: >>> 1.dscp = 46 >>> 2.pbit = 6 >>> Struct[47] has 2 members: >>> 1.dscp = 47 >>> 2.pbit = 0 =============================================================== › Using these MOs we are able to map out the quality of service properties defined in the RBS Transport Network Configuration (39/1553-HSC 105 50/1) GTP-U - User Plane teid, ip address, port teid, ip address, port GTP-U UDP IP Data Link GTP-U UDP IP Data Link Physical Physical S1-U › GTP-U is used as the User Plane protocol over the S1 and X2 interfaces (defined in 29.060) › GTP-U carries the end user data by forming tunnels towards the core network S-GW (transports user IP payload) › IP fragmentation is to be avoided (MTU size should be set correctly) › Configuration aspects on the RBS are minimal (no MO models this layer) Transport Network performance › Transport Network performance visibility is available via the GigaBitEthernet MO and IpAccessHostEt pm counters – Moshell> pmom GigabitEthernet|IpaccessHostEt › The metrics include (singleton, sample and statistical metrics): – GE Link Ingress/Egress Average usage – GE Link Ingress Frame Error Ratio – IPv4 Ingress/Egress Packet discard ratio L2 observability L3 observability › These TN metrics can help identify: – under-dimensioned GE links (i.e. highly utilised links) – transport problems/errors over GE links (high frame discard ratios) – IPv4 packet discard issues (queue capacity, errored packets, etc) Transport Network Performance Metrics (44/1553-HSC 105 50/1) Transport Network performance › Additional TN Performance Management counters are available › SCTP (Signalling transport for S1AP and X2AP) – Sent and received data/control chunks – Dropped chunks (buffer overflows) – Checksum errors › Synchronization (SoIP related counters) – Provides the highest delay variation counters for the active IP sync reference – Calculated in terms of the best x percentage sync frames experienced during a 100 second window (result in microseconds) – The percentages include 1, 10, 50 % › IpInterface (Signalling, payload and sync) – Failed Pings to default routers (RPS) – Discards, header and IP address errors RBS Counters via COLI $ EtHostMo_getPmCounters -h 1 -t 1 › Detailed IP counters are available from the transport interface (TN-A / TN-B) PM counters for host with hostFroId 1 ipAddrErrors=0 ipInDelivers=23048 ipInDiscards=-2 ipInHdrErrors=0 › IP layer (via IpAccessHostEt MO) – EtHostMo_getPmCounters -h 1 -t 1 ipInReceivedOctets=-2 ipInReceives=321055 ipInUnknownProtos=0 ipNumFailedAt=-2 ipOutDiscards=-2 › The counters show: – Protocol errors – Ingress/Egress Discards – UDP protocol errors – ICMP details – Fragmentation details ipOutRequests=321945 ipOutRequestOctets=-2 ipReasmReqds=0 ipReasmOKs=0 ipReasmFails=0 ipFragOKs=0 ipFragFails=0 ipFragCreates=0 ipPortUnreachable=0 udpInDatagrams=0 udpInErrors=0 udpNoPorts=-2 udpOutDatagrams=0 icmpInDestUnreachs=0 <truncated> RBS Counters via COLI › RBS COLI counters show detailed performance issues – S1AP/X2AP (SCTP MO) – sctphost_stat -assoc -all › SoIP (Synchronization MO) – nssinfo tupm $ sctphost_stat -assoc -all sctphost_stat - START. |-------------------------------------------------------| |----------------------- SCTP HOST ---------------------| |RpuId: 17 |SctpInstId: 0 |Base State: BASE_RUN |Host State: A|C|R|X|IA |Ext. client: CONNECTED |Alarm Timer: NOT RUNNING |---------------- SCTP ASSOCIATION 96 -----------------| |-------------------------------------------------------| $ nssinfo tupm |----------------Statistic (assoc level)----------------| ****** NSS TUM2 tu_pm related data ****** | [ID 7]: SCTP_STAT_SENT_CHUNKS, Count: 1| | [ID 8]: SCTP_STAT_REC_CHUNKS, Count: 1| | [ID 9]: SCTP_STAT_OUT_OF_ORDER_SC, Count: 0| | [ID 10]: SCTP_STAT_OUT_OF_ORDER_RC, Count: 0| | [ID 12]: SCTP_STAT_RETRANS_CHUNKS, Count: 0| | [ID 13]: SCTP_STAT_SENT_CC, Count: 16264| PMfroId : 1 PMfroType : 66817 granularityPeriod : 900 suspectFlag : 0 par_pmMDVCounter : 186 | [ID 14]: SCTP_STAT_REC_CC, Count: 16264| par_pmHDVB1Pct : 0 | [ID 15]: SCTP_STAT_FRAG_USER_MSG, Count: 0| par_pmHDVB10Pct : 1 | [ID 16]: SCTP_STAT_REAS_USER_MSG, Count: 0| par_pmHDVB50Pct : 5 | [ID 17]: SCTP_STAT_SENT_PACKAGES, Count: 16265| | [ID 18]: SCTP_STAT_REC_PACKAGES, Count: 16265| ****** END ****** <truncated> Transport Network tracing › RPS – $ appdh info / $ appdh rps <ip inter> – Use Wireshark to see ICMP › QCI, ARP, AMBR values – te e bus_send bus_receive S1AP_ASN (e.g. InitialContextSetupRequest) › SCTP – $ te e all cpxSctpIC – $ te e all Scc_SctpHost_proc › Synchronization (SoIP using NTP) – te e trace7 NSS_CBM_TUM2_TUREG – $ nssinfo all Wireshark - Protocol capture S6a HSS MME S1-MME S11 FTP Server eNB S1 IP Backbone S1-U S-GW S5/S8 P-GW Internet Wireshark is an open source protocol analyser. It provides excellent support of the 3GPP LTE protocols. S1 capture via Wireshark Will filter all data and only display S1AP protocol messages QCI = 9 S1 capture via Wireshark › Example S1 trace with user and control plane payload: S1-MME SCTP S1-U GTP-U L2 QoS Pbit = 6 L3 QoS DSCP = 46 SoIP using NTP Transport Summary › LTE RAN transport topology is very flexible – a mixture of L2/L3 topologies could be implemented (could complicate analysis) › Basic transport network redundancy is provided (in terms of L3 routers) › Transport network dimensioning should be taken into account as statistical gains are used (backhaul peaks should be known) › QoS (Radio and Transport) is essential for proper network operation and should be implemented throughout the network (i.e. in L2/L3 nodes as well) › Basic performance management is provided in initial LTE RAN releases › Additional observability can be secured through protocol analysis E2E Analysis e2e analysis › End-2-end analysis of throughput is required when end users report throughput issues that are not readily seen in the LTE RAN › End user throughput investigations/analysis is done at the UE/Server side › Typical user payload will use the TCP transport protocol › Layer 3 IP configuration aspects are not covered › The UDP transport protocol is not covered lte end user Protocol stack Telnet, FTP, TFTP, HTTP, SNMP, ….. Application Layer Port Number BGP Transport Layer OSPF RIP EGP TCP UDP IMCP IGMP Protocol Number Internet Layer ARP IP RARP Type Code Data Link Layer PDCP, GTP-U TCP at a glance › TCP will be the most used transport layer protocol (email, ftp, browsing, etc) › TCP offers the following features: – Stream data transfer › TCP offers a contiguous stream of segments for applications – Reliability › Sequence numbers used by sender that expects positive acks (or retransmit) › Receiver uses sequence numbers to rearrange segments (remove duplication) – Flow control › Receiver indicates the number of bytes it can receive (to sender) – Multiplexing › Achieved through the use of ports – Logical connections › Each connection is identified by the pair of sockets used (in receiver & sender) – Full duplex operation › TCP provides concurrent data streams in both directions TCP Header Provides reliability Indicates application rwnd - what the receiver (sender of this segment) is willing to accept TCP Operation › TCP operates in three distinct phases: – Connection establishment – Data transfer – Connection termination › TCP is known as a sliding window protocol › Two windows are defined: – rwnd - receive window, advertised/offered by the receiver – cwnd - congestion window, calculated by the sender › The TCP sender window is defined as the minimum of rwnd and cwnd › AIMD behaviour (additive increase / multiplicative decrease) – cwnd = cwnd + MSS*(MSS / cwnd) – cwnd = cwnd / 2 › Sliding window in action: Sliding Window Animation TCP Operation Cont. › TCP congestion control: – Slow start › exponential growth of cwnd › used in cold/initial start and after a timeout – Congestion avoidance › linear increase after congestion experienced › congestion indicated by timeouts and duplicate acks – Fast retransmit › sender quickly determines congestion and retransmits › attempts to avoid timeouts - and hence slow start – Fast recovery › enter congestion avoidance rather than slow start › Senders use a retransmission timeout (RTO) based on RTT TCP Congestion Control Congestion Window 3rd DUPACK Fast retransmit 3rd DUPACK Time out 3rd DUPACK Pipe Capacity Slow start threshold reached Fast recovery Congestion Avoidance Slow Start ssthresh on congestion t TCP Performance › Bandwidth Delay Product (BDP) - data required to fill the TCP pipe: – Bandwidth of link (bytes per sec) * Delay (sec) = amount of data in transit to fill pipe › Example: A T1 (1.5 Mbps) over a satellite connection with RTT = 500ms – (1,500,000 bits per sec / 8 bits per byte) * (0.5 sec) = 93,750 bytes – With a rwnd of 65,535, performance is 65,535/93,750 = ~70% or 1.05 Mbps – Hence, the BDP requires more transit data, and this is achieved with an increase in the rwnd (the TCP scaling feature, RFC1323, can provide this increase) › TCP Windowing (rwnd) and RTT limit the achievable throughput as follows: – › Hence, large receive windows and small RTT are desired Typical LTE RTT TCP Tuning (Client side) › TCP performs differently on different Operating Systems (there have been many variations on the TCP congestion control algorithm for instance) › On Windows Vista: – autotuning = highlyrestricted › netsh interface tcp set global autotuninglevel=highlyrestricted – autotuninglevel = restricted › netsh interface tcp set global autotuninglevel=restricted › On Windows XP: – SP2+: follow this Windows XP TCP configuration KO – Pre-SP2: Use the DrTCP application › On Linux (depends on the kernel) – Follow this TCP Tuning Guide for kernels 2.4 -> 2.6 (i.e. 2.6.20+ are covered) LTE RAN TCP Behaviour › Processing (handovers, buffering, delays, scheduling, retransmissions) in the RBS can affect TCP operation › Affect of incorrect TCP receive window sizes: – Lower than BDP -> › Packet loss can lead to (retransmissions, dropped in RBS, etc): – TCP retransmissions and delays – Send rate (throughput) reduced up to 50%, then a linear increase until next drop or max TCP rate reached › TCP timeouts can lead to: – TCP fallback into slow start LTE RAN TCP Enhancements › LTE RAN future adaptations to enhance end user TCP behaviour: – AQM (Active Queue Management) › Uses the TCP congestion avoidance algorithm › Limits the cwnd by selectively dropping packets – Result: smaller cwnd = smaller buffered data and delay – Data forwarding at intra LTE handover (X2) › Zero packet loss at handover › Provides higher data rates for mobile users – Result: reduces the risk of TCP fallback into slow start Analysing e2e traffic › The following section uses the Wireshark application to inspect e2e traffic (i.e. FTP download from UE) › Two scenarios are used to highlight how Wireshark can perform detailed analysis on network traffic › The goal is to show how e2e analysis can be performed from a UE perspective › The following shows analysis of downlink FTP data session towards a UE Throughput Results › For DL throughput, select a packet from Server to UE, and select Statistics -> TCP Stream Graph -> Throughput Graph Scenario A Scenario B Throughput Results Cont › Two other ways to quickly determine the UE throughput are via Statistics -> Summary and Statistics -> IO Graphs Average added for clarity Scenario A Time Sequence Graphs › A great insight into the TCP performance/behaviour is found with the Statistics -> TCP Stream Graph -> Time-Sequence Graph (Stevens) Scenario A Scenario B TCP Flow Graphs › TCP flows (Statistics -> Flow Graph) are useful in isolating a particular TCP conversation Scenario B Additional TCP Analysis › Wireshark’s Expert Info (Analyse -> Expert Info) provides a better display of uncommon or notable network behaviour: Scenario B Analysis Suite › Wireshark is a powerful tool to inspect e2e network behaviour › Other tools can be used in conjunction with Wireshark to offer a complete e2e analysis suite: – iperf › works in client/server modes › allows UDP and TCP payload to be injected into the network › useful to see link capacities with UDP (max DL/UL throughput) › identifies possible TCP bottlenecks – Netpersec › Offers real-time display of throughput – IXIA Chariot Endpoint › commercial product to test IP networks › provides automated application level analysis (HTTP, VoIP, etc) › provides detailed results/statistics of performance e2e Summary › The main transport layer (OSI L4) protocol used for network services (internet) will be the Transmission Control Protocol (TCP) › TCP offers end user application reliable data delivery, flow control and good bandwidth utilisation › The LTE RAN will implement features/functions that provide better inter-work with the TCP protocol › Wireshark is a powerful open source network protocol analyser that can offer advanced e2e throughput investigations using inbuilt functions › Additional tools (like iperf) support the complete e2e analysis Summary Summary › Throughput troubleshooting in the LTE RAN can be broken into domains › The Radio domain includes many uplink and downlink specific areas of analysis and can also be utilised to find eNB external problems › The Transport domain analysis focus’ on northbound IP network › The e2e domain analysis looks at end user throughput concerns More Information › LTE PLM Troubleshooting Wiki: https://plmlte.rnd.ki.sw.ericsson.se/lte_trsh_wiki/index.php?n=UseCases.UseCases › LTE Tutorial: http://utran01.au.ao.ericsson.se/flowfox/lteman.html › LTE Observability: http://utran01.au.ao.ericsson.se/flowfox/lteobservability.html › PLM LTE Toolbox in the field: https://ericoll.internal.ericsson.com/sites/PLM_LTE_Toolbox_in_field/default.aspx › LTE FlowFox: http://utran01.au.ao.ericsson.se/flowfox/index.php#lteflowfox › LTE SW Deployment and Support EKB Community: https://knowledgebase.internal.ericsson.com/LTE_RAN/ › 3GPP 36-series specifications (TS36.213 in particular for Radio Analysis section): http://www.3gpp.org/ftp/Specs/html-info/36-series.htm › LTE L11A GA (R9AJ) System Constants list http://cdmweb.ericsson.se/WEBLINK/ViewDocs?DocumentName=1%2F19059-HRB105500&Revision=PE2-10 › LTE RAN Data Collection Guideline: http://cdmweb.ericsson.se/WEBLINK/ViewDocs?DocumentName=60/1543-LZA7016004&Latest=true