#CiscoLive Synchronizing 5G Mobile Networks Network Based Timing Shahid Ajmeri, Sr. Product Manager, Provider Connectivity Group BRKSPG-3050 #CiscoLive Cisco Webex App Questions? Use Cisco Webex App to chat with the speaker after the session How 1 Find this session in the Cisco Live Mobile App 2 Click “Join the Discussion” 3 Install the Webex App or go directly to the Webex space 4 Enter messages/questions in the Webex space Enter your personal notes here Webex spaces will be moderated by the speaker until June 9, 2023. https://ciscolive.ciscoevents.com/ciscolivebot/#BRKSPG-3050 #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 3 Disclaimer: Use of the terms “master” and “slave” in this presentation is ONLY in association with the official terminology used in industry specifications and standards (for example IEEE1588, ITU-T SG15/Q13 specifications, eCPRI, ORAN-Alliance and so forth), and in no way diminishes Cisco’s commitment to promote diversity, equity, and inclusion. We will be sharing our efforts to remove this language from our products, content and culture at Cisco. If at any point you feel uncomfortable, please feel free to exit the session for a couple of minutes and then rejoin. You can also reach out to us directly with questions, or requests for more information on our approach in a format that works for you. BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 4 Key Abbreviations APTS Assisted Partial Timing Support O-DU Open Distributed Unit (a)BMCA (alternate) Best Master Clock Algorithm O-RU CA Carrier Aggregation PRTC Open Radio Unit Primary Reference Time Clock CoMP PTP cTE Coordinated Multi-point Constant Time Error dTE Dynamic Time Error SyncE EEC Ethernet Equipment Clock T-BC Synchronous Ethernet Telecom Boundary Clock eEEC Enhanced EEC T-GM Telecom Grand Master ESMC Ethernet Synchronization Message Channel T-TC Telecom Transparent Clock eSyncE T-TSC FTS Enhanced SyncE Full Time Support Telecom Time Slave Clock Time Alignment Error GNSS Global Navigation Satellite System TDEV MITM Man in the Middle TE MTIE Maximum Time Interval Error TSN O-CU Open Centralized Unit UTC PTS TAE #CiscoLive BRKSPG-3050 Precision Time Protocol Partial Time Support Time Deviation Time Error Time Sensitive Networking Universal Coordinated Time © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 5 Agenda • Introduction • New Concepts and Technologies • Redundancy, Resiliency and Monitoring • PTP Security in 5G RAN • Design Guidelines • Summary #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 6 Drivers of Precision Timing Emergencies Location Accuracy Positioning Correlation AR/VR/MR, V2V, Drones, Aviation, Advertising Behavior and Event analysis IT Operations, Monitoring, Log analysis Syntonization Ensuring two nodes are synchronized on correct rate or frequency Control Behavior and Event analysis Sensor data processing Efficiency Emergency response, Tracking, Defense Security / Trust Event Detection Predict Future Events Improve Bandwidth Channel Processing, KPIs 4G LTE / 5G Radios WiFI6 / WiFi 7 AI/ML Machines Avoid Casualties Proving when event happened. Detection and Response IOT, Smart Cities, Education, Healthcare TDM, Optical Networks Precision and Traceability #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 7 Time Synchronization across Architectures Performance Measurement / Service Assurance Cable Network Timing xHaul Network Timing DOCSIS DU/CU RPD SR PM for accurate latency measurement Pre-Agg / Agg RPD RPD Residential Wholesale Providers Timing 3rd Party / Metro Links APTS Feature Backup Timing DU/CU GPON DOCSIS Pre-Agg / Agg Timing as a Service RPD RPD OLT OLT APTS: Assisted Partial Timing Support Public or Private 5G Metro Transport #CiscoLive BRKSPG-3050 Cable / GPON Transport © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 8 Time Synchronization in 5G Networks Cell Synchronization Requirements • Transmitter power ON/OF • Change Transmit / Receive modes • Air propagation time etc. TDD TAE 3µs Coordinated Transmission and Reception Increase throughput with Carrier Aggregation, Dual connectivity • Improve performance with coordinate multipoint operation etc. • TX Diversity / NB-IoT TAE <65ns Frequency Band A Frequency Band B Intra Band Contiguous <130ns FR2, LTE <260ns FR1 Frequency Band A Frequency Band B Intra Band Non-Contiguous <260ns (FR2, LTE) 3us (FR1) Frequency Band A Frequency Band B Inter Band CA <260ns (LTE) 3us (NR) Application Requirements Positioning and Tracking Time-sensitive networking: Robot control or Autonomous Vehicles • Extended Reality etc. • • Positioning TAE <100ns 3GPP TS 36.104, 38.104 BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public TAE : Time Alignment Error #CiscoLive 9 Timing Distribution in Radio Access Network GNSS per Cell Site PRTC/T-GM Network Based Timing PRTC/T-GM PRTC/T-GM T-BC Transport Network T-BC Transport Network PRTC: Primary Reference Timing Clock, T-GM: Timing Grand Master, T-BC: Telecom Boundary Clock, GNSS: Global Navigation Satellite System #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 10 Deployment Options T-BC G.8275.1 Full Path Support T-BC PTP Aware Network T-BC PTP Aware Network PTP Aware Network PRTC / T-GM T-BC-P / T-TSC-P T-BC-P G.8275.2 Partial Timing Support PTP Unaware Network PTP Aware Network PTP Unaware Network PRTC / T-GM PRTC Assisted Timing T-BC-P Assisted Partial Timing Support PTP Unaware Network PTP Aware Network PTP Unaware Network PRTC / T-GM T-BC-A / T-TSC-A #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 11 5G Synchronization Requirements Requirement 1: Absolute Time At Network level: • Phase Accuracy: 1.1 µs • Frequency Accuracy: 16pbb 1 2 3 N a 1 2 3 N b Max|TE| < 1.5µs for entire chain b1 Nb Requirement 2: Relative Time At Air interface Phase: TAE 3µs Frequency: 50pbb b2 b3 a1 1 2 N 3 a2 Time Error accumulation up to node N Max|TE| < 1.5µs for entire chain PRTC/T-GM T-BC Max |TE| < 260ns Inside synchronization Cluster Na Nc c1 c2 Relative Air Interface TAE within cluster 130ns / 260ns c3 T-TSC / gNB #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 12 ORAN Fronthaul Architecture Options C1 C2 Transport Network O-DU Network Timing/SyncE Backup Transport Network Dark Fiber O-RU O-DU Network Timing/SyncE Backup Timing / SyncE O-RU Fronthaul Timing / SyncE C4 C3 Timing / SyncE Transport Network Transport Network O-DU Fronthaul O-DU O-RU Network Timing/SyncE Backup Network Timing/SyncE Backup #CiscoLive BRKSPG-3050 Fronthaul O-RU Timing / SyncE © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 13 Time Budget Example : ORAN C3 Architecture A/B C O-DU PRTC/T-GM O-RU T-BC T-TSC T-BC T-TSC O-RU Transport Network Relative Air Interface TAE within cluster 260ns Fronthaul T-BC T-BC Network Timing/SyncE Backup PRTC/T-GM to O-DU 1.1µs 1) Time Error at Radio Relative Time Budget on Fronthaul Transport 100ns to 190ns 2) Time Error at UNI of O-RU 3) Relative Time-budget on transport a) Error at Radio Elements |TERE| = 20ns O-RU |maxTE| = |TERE| + T-TSC |TE| 260ns – 2 x O-RU |maxTE| b) Regular O-RU with Class B T-TSC T-TSC |maxTE| = cTE + dTE = 60ns a) Regular O-RU = 80ns a) For Regular O-RU 260ns – 2x80 = 100ns i.e. 50ns per link b) enhanced O-RU = 35ns b) enhanced O-RU 260ns – 2x35 = 190ns i.e. 95ns per link c) enhanced O-RU with Class C T-TSC T-TSC |maxTE| = 15ns #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 14 Type of Clocks and Network Budget T-BC “1” PRTC-A 100ns PRTC-B 40ns ePRTC 20ns Link Asymmetric Compensation 380ns 590ns 10ns T-BC “n” 20ns Random Network Variations 10ns 200ns D C G.8271.1 Network Reference Points A/B Class B T-BC “21” 21 nodes Short Term Holdover 20ns 10ns 250ns 150ns End Application Class C 21 nodes 1.1µs Network Budget 30ns 1.5µs End-to-End Budget T-BC / T-TSC / T-TC cTE dTE (MTIE) Max |TE| dTE (high pass filter) Class A (with SyncE) 50ns 40ns 100ns 70ns Class B (with SyncE) 20ns 40ns 70ns 70ns Class C (with eSyncE) 10ns 10ns 30ns (T-BC) Under study for T-TC Under Study Class D (with eSyncE) *no T-TC Under Study Under Study 5 ns (low pass) * Under Study * Measured with first order filter of 0.1Hz #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 15 Recommended Approach for 5G Network Based Timing GNSS per Cell Site Centralized PRTC provide better solution to meet relative time-error • In theory, PRTC-A receivers at cell site will not meet relative time budget, PRTC-B or ePRTC may help. • Network node with Class C T-BC performance are required. 200ns error from PRTC alone +100ns Provide better Resiliency and Redundancy -100ns PRTC/T-GM BMCA algorithms for best master selection Virtual port: for GNSS receiver redundancy Assisted Partial Timing support (APTS), with backup PTP, for better accuracy and holdover on partial timing aware networks and so forth • • • PRTC/T-GM Transport Network Better antenna installations with clear satellite visibility Reduced cost • • Less antenna and PRTC/GNSS installations Better operations BMCA: Best Master Clock Algorithm #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 16 New Concepts and Technologies To meet 5G Synchronization Requirements GNSS Receiver Performance Impacting Factors Ionosphere Transmission delays and Errors Troposphere Ionospheric transmission delays and errors • • • Satellite visibility Antenna siting Cable delay correction • • #CiscoLive Multi-path Interference (from other sources) BRKSPG-3050 Correction data available to the algorithm © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 18 Ionosphere Errors L1 Ionosphere L1 L1 Single Band Antenna Antenna Cable PRTC-A GNSS = 100ns The biggest source of time error in PRTC-A receivers is disturbed signal propagation through the ionosphere which: Solar activity is expected to be much higher in next 4 years, which will impact PRTC-A performance • Has 24 hours diurnal cycle depending on the rotation of earth, with minimum delay at night. • Depends on space weather, resulting from changes in solar activity (11-year cycle) #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 19 PRTC-A Performance Not going to be accurate enough for 5G services Illegal Jammers Antenna installation are not always in open skies. Satellite visibility is limited in built-up areas Solar activity is expected to be more active, local and instantaneous errors could impact GNSS receiver #CiscoLive BRKSPG-3050 Off the shelf Jammers Single band limits antijamming / anti-spoofing capabilities © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 20 PRTC-B Receivers Necessary to meet 5G requirements L1 L2 L1 L2 L1 L2 Ionosphere Dual Band Antenna Antenna Cable PRTC-B GNSS = 40ns Stability • PRTC-B receivers are Dual-band receivers • Measures ionospheric anomalies and models these effects better to improves overall stability of the clock #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 21 ✓ Multi-band support for better accuracy • Improves traceability to UTC accuracy ✓ Better security against Anti-Jamming and Anti-Spoofing attacks ✓ Improves overall network time-budget requirements PRTC-B Benefits Cisco Solution • 8000 Series platforms ✓ Addresses “Relative Time Budget” within cluster of radios ✓ Helps meet stringent position tracking and location accuracy across Private 5G and Public 5G deployments • Public 5G: • Carrier Aggregation, Dual Connectivity, CoMP • TSN, Power Grid, Factory Automation, Robotics, Drone • Enterprise: • Metaverse: • AR / VR use cases • AI / ML and Analytics BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 22 eSyncE / eEEC (G.8262.1) Mandatory for Class C Clocks and 5G RAN • 5x improvement to frequency synchronization performance Stability EEC (G.8262) eEEC (G.8262.1) Wander Generation MTIE: 40ns @ 0.1s TDEV: 3.2ns @ 0.1s MTIE: 7ns @ 0.1s TDEV: 0.64ns @ 0.1s Noise Transfer Filtering EEC (G.8262) eEEC (G.8262.1) Clock Bandwidth 1-10 Hz 1-3 Hz Holdover EEC (G.8262) eEEC (G.8262.1) Short term phase Transient Response @ Const Temp 120 ns at Initial step 10 ns at Initial step Initial response to loss of sync reference 50 ns/sec frequency offset, plus 1.16 X 10-4 ns/s2 drift 10 ns/sec frequency offset, plus 1.16 X 10-4 ns/s2 drift • Better phase/time synchronization performance for SyncE-assisted PTP • Improved time noise transfer performance • Phase jump due to noise transfer from SyncE to PTP is low-pass filtered • Improved holdover performance • Longer chain or stable performance for long time @ 1.5us performance MTIE: Maximum Time Interval Error, TDEV: Time Deviation #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 23 ESMC and Enhanced ESMC (G.8264) Enhanced ESMC Include new TLVs containing: • • • • QL Which Quality I should use? PRC Originator Clock-id New SSM Codes SyncE steps removed from nearest SSU/PRC eSyncE steps removed from nearest SSU/PRC QL PRTC QL Improves Solution: 1. Allow higher accuracy clocks 2. Separate counts of EEC/SEC and eEEC/eSEC hops from originator source 3. Traceability and Monitoring 4. Identify the source node • Detect Timing Loops ESMC: Ethernet Synchronization Message Channel #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 24 ESMC and Enhanced ESMC (G.8264) 2. New flags helps in traceability: 1. New Hop Counters for EEC and eEEC clocks clockIdentity : Node1 Count of EEC clocks: 0 Count of eEEC clocks: 1 Mixed EEC/eEEC: No Partial chain bit: No 1 clockIdentity : Node1 Count of EEC clocks: 0 Count of eEEC clocks: 2 Mixed EEC/eEEC: No Partial chain bit: No 2 • • • Do all nodes in the chain support eSyncE? Do all nodes in the chain support eESMC? Was the TLV generated in the middle of the chain? clockIdentity : Node1 Count of EEC clocks: 0 Count of eEEC clocks: 3 Mixed EEC/eEEC: No Partial chain bit: No 3 clockIdentity : Node1 Count of EEC clocks: 0 Count of eEEC clocks: 4 Mixed EEC/eEEC: No Partial chain bit: No 4 Flags: 5 Extended QL TLV Is chain mix of SyncE and eSyncE = 0 = NO The chain has non eESMC nodes = 0 = NO clockIdentity : Node1 Count of EEC clocks: 0 Count of eEEC clocks: 5 Mixed EEC/eEEC: No Partial chain bit: No Original QL TLV eEEC and eESMC EEC and eESMC EEC and ESMC #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 25 ESMC and Enhanced ESMC (G.8264) 2. New flags helps in traceability: 1. New Hop Counters for EEC and eEEC clocks clockIdentity : Node1 Count of EEC clocks: 0 Count of eEEC clocks: 1 Mixed EEC/eEEC: No Partial chain bit: No 1 clockIdentity : Node1 Count of EEC clocks: 0 Count of eEEC clocks: 2 Mixed EEC/eEEC: No Partial chain bit: No 2 • • • Do all nodes in the chain support eSyncE? Do all nodes in the chain support eESMC? Was the TLV generated in the middle of the chain? clockIdentity : Node1 Count of EEC clocks: 1 Count of eEEC clocks: 2 Mixed EEC/eEEC: Yes Partial chain bit: No clockIdentity : Node1 Count of EEC clocks: 1 Count of eEEC clocks: 3 Mixed EEC/eEEC: Yes Partial chain bit: No 4 3 clockIdentity : Node1 Count of EEC clocks: 1 Count of eEEC clocks: 4 Mixed EEC/eEEC: Yes Partial chain bit: No 5 EEC and eESMC Flags: Extended QL TLV Is chain mix of SyncE and eSyncE = 1 = YES The chain has non eESMC nodes = 0 = NO Original QL TLV eEEC and eESMC EEC and eESMC EEC and ESMC #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 26 ESMC and Enhanced ESMC (G.8264) 2. New flags helps in traceability: 1. New Hop Counters for EEC and eEEC clocks clockIdentity : Node1 Count of EEC clocks: 0 Count of eEEC clocks: 1 Mixed EEC/eEEC: No Partial chain bit: No 1 clockIdentity : Node1 Count of EEC clocks: 0 Count of eEEC clocks: 2 Mixed EEC/eEEC: No Partial chain bit: No 2 • • • Do all nodes in the chain support eSyncE? Do all nodes in the chain support eESMC? Was the TLV generated in the middle of the chain? clockIdentity : Node4 Count of EEC clocks: 1 Count of eEEC clocks: 1 Mixed EEC/eEEC: Yes Partial chain bit: Yes No Extended TLV 3 4 clockIdentity : Node4 Count of EEC clocks: 1 Count of eEEC clocks: 2 Mixed EEC/eEEC: Yes Partial chain bit: Yes 5 ESMC, EEC Flags: Extended QL TLV Is chain mix of SyncE and eSyncE = 1 = YES The chain has non eESMC nodes = 1 = YES Original QL TLV eEEC and eESMC EEC and eESMC EEC and ESMC #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 27 ✓ eSyncE is mandatory for Class C timing eSyncE and eESMC ✓ eSyncE provides better performance, stability and holdover to the clock ✓ Enhanced ESMC improves operations • Avoid SyncE loops • Provides hop counts • improves traceability and Monitoring BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 28 Compute Based Timing Most popular outdoor deployments are C1 architecture Linux PTP Project Status Check: • Profile configuration is static • • • Limited or no support for Remote Management • • • S-plane monitoring Phc2sys and ts2phc state monitoring Operations / Non-compliance • • • Cisco NCS540 • Linux PTP Project – Timing NIC • • • • Any change require service restart Impacts radio operations • • • ptp4L: Synchronize clock using PTP protocol ts2phc: Synchronize clock to external signal e.g. 1PPS phc2sys: synchronize NIC clock to system clock pmc: PTP management client #CiscoLive Not fully compliant to ITU-T G.8275.1/G.8275.2 Not compliant to G.8264 and G.781 Doesn't comply to ORAN WG4 Specification • Test Cases Section 3.3 on S-plane compliance ptp4l uses s0/s1/s2/s3 status instead of “locked”, “hold over”, “freerun” • Doesn't update status in “announce” messages No Support for Primary to Backup switching No Support for profile interworking No support for APTS or Virtual Port – no resiliency or redundancy for timing BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 29 ✓ Industry needs to focus on contributing and improving Linux PTP project Compute Based Timing ✓ For 5G architecture, network-based timing provide most advanced solution ✓ Network-based timing could help reduce complexity and cost on compute BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 30 Redundancy, Resiliency and Monitoring Redundancy for Grand Master Use Case: Relative Timing within Radio Cluster, Fronthaul and Private 5G Requirement: • Set of RUs connected to a common DU • All radios and DUs need to be synchronized by common PRTC for better accuracy and to manage relative timing within cluster • T-GM 200ns error from PRTC alone O-DU T-GM Seamless network failover in case of PRTC failure #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 32 Redundancy for Grand Master Approach - Virtual Port (VP) • Note: T-GM can only distribute clock (all ports are master ports), can not receiver it. But T-BC can. • Virtual Port (G.8275.2 Annex B) provides electrical frequency and phase inputs to T-BC • Virtual Port gives flexibility to configure node as ”boundary clock” and run BMCA to select best time source – Local GNSS or remote grandmaster GNSS GNSS VP VP S BC Router A Router A has better clock quality M Router B runs BMCA and selects router A as primary clock #CiscoLive BRKSPG-3050 S S BC Router B © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 33 Redundancy for Grand Master Use cases: Relative Timing within Radio Cluster, Fronthaul and Private 5G Solution: Primary • • Configure Primary and Secondary PRTC priority Configure each node as T-BC with VP (virtual port) to select priority for local vs remote GNSS • Primary PRTC distributes clock to all nodes – O-RUs, Routers, O-DUs • In case of Primary PRTC failure, BMCA will auto select secondary PRTC T-BC with VP Common clock source for all Radios within a cluster Secondary O-DU T-BC with VP #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 34 Resiliency with backup PTP Use Case: Last mile over 3rd party links O-DU G.8275.2 Requirement: G.8275.2 • Set of RUs connected over 3rd party links / 3rd party network that doesn’t support timing 3rd Party Transport #CiscoLive BRKSPG-3050 PTP un-aware network doesn't guarantee timing performance © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 35 Resiliency with backup PTP Approach: Assisted Partial Timing Support (APTS) • APTS function allows T-BC to select local GNSS as the primary source; and continuously measures the time offset compared to that recovered from the back up PTP signal. • In case of GNSS failure, T-BC will failover to back up PTP and removes the observed offset to maintain time accuracy during GNSS failure. • APTS configuration requires Virtual Port (G.8275 Annex B) support GNSS Measure offset from backup PTP clock. In case of local GNSS failure, maintain clock accuracy with backup signal VP S BC Router A M S Backup PTP from remote router #CiscoLive BRKSPG-3050 BC Router B © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 36 Resiliency with backup PTP Use Case: Last mile over 3rd party links O-DU G.8275.2 Solution: • Connect local GNSS at each site to accurately synchronize Radios • Use back up PTP (G.8275.2) from network to maintain performance during local GNSS failure • When local GNSS is down, accuracy and stability of the clock depends on PTP clock quality G.8275.2 PTP as backup 3rd Party Transport #CiscoLive APTS BRKSPG-3050 APTS APTS © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 37 ATPS Deployment Guidelines The clock locks to the PTP reference, and the dTE is larger, and cTE could be also larger, due to the asymmetry caused by PTP unaware network. Although the clock can compensate it in advance, some remaining cTE could be still present. Time Error (ns) The clock is in holdover within a short time, decided by BMCA algorithm Lock to local GNSS Lock to local GNSS • APTS is considered mainly for “Partial timing aware network” • Backup Timing should be available as part of G.8275.2 profile G.8275.2 over partial aware network is not reliable. Please ensure G.8275.2 quality is good, to get max benefits of APTS • Avoid multi-profile with APTS 22 ns (transient for switching & residual PTP cTE) + • 15 ns (tranfser PTP gain) + X ns (generation XO PTP dTE) • Time Local GNSS is lost, clock goes into holdover and generates a first transient The clock selects the PTP reference, and changes from holdover to lock, and generates a second transient APTS can correct STATIC time error, dynamic error is still un-controlled • Only good for last measurement GPS is restored and selected by the clock #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 38 Holdover with SyncE Assistance Use Case: Longer holdover • Holdover is either oscillator only, or assisted with external frequency (SyncE) • Staying within 1.1 µs is only possible for several hours when relying on local oscillator Loss of SyncE Zero ns Realigned 1.1 µs Loss of PTP (11 mins) 5 Hours Reacquired 10 Hours #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 39 Network Monitoring for Improved Resiliency Monitoring Alternate Master Time Information • Allows PTP monitoring of alternate path • Calculates PTP offset between PTP received from primary path and alternate link • If offset crosses define threshold, monitoring alarm is generated • • This alarm doesn’t trigger BMCA The functionality helps monitors network health • Fiber asymmetry between links, network failures and so forth Use Case 1: Passive port monitoring Master port Use Case 2: Master port monitoring Passive Port ITU-T G.8275.1 Annex G #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 40 ✓ VP is for timing redundancy • PRTC Redundancy • G.8275.1 based full path support networks ✓ APTS is for better timing resiliency Redundancy, Resiliency and Monitoring • Local GNSS for timing accuracy • Backup PTP for maintain performance during GNSS failure ✓ Holdover with SyncE assistance • Improves overall performance ✓ Cisco Solution support mature PTP monitoring and management solution • SNMP MIBs: • • • CISCO-PTP-MIB CISCO-NETSYNC-MIB CISCO-GNSS-MIB • Yang / Telemetry Support BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 41 PTP Security in 5G RAN Timing Security Attacks Attack Impact Attacker Type Internal False Time Accuracy degradation External DOS MITM Packet Injector Changing Clock Parameters X X Spoofing X X X Replay Attack X X X Rogue Master Attack X X X Changing and removing Messages Packet Delay Manipulation X X X MITM X X X X Packet Injector DOS attacks X X X X X Cryptography Performance attacks X X X X X X X X X GPS Spoofing / Jamming X #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 43 MACsec and Security for Timing • MACsec is a L2 port-based security mechanism • Either encryption + integrity checking OR only integrity checking (IEEE 802.1AE) • Maybe mechanisms to choose whether to send secured/unsecured on a port • Either asymmetry or dynamic time error can arise through: • Difference in time to encrypt/decrypt messages • Selection of receive/transmission timestamping point • Differences in time to encrypt/decrypt subsequent messages MAC DA MAC SA SEC Tag Ether -type Payload ICV FCS MAC DA Encrypted Data MAC SA Ethertype Payload FCS Original Packet Integrity Protected #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 44 PTP over MACSec Deployment Challenges PTP PHY detects PTP packet and Provides Timestamp for “Follow Up” message” MACSec PHY Timestamp is ideally done when packet hits wire – at PHY. PTP MACSec PHY 1-Step Clock 2-Step Clock • PHY based time-stamping doesn’t work for 1-step clocks • Timestamp can be generated by measuring “Sync” packet delay at the egress, and sub-sequent “Follow Up” message can carry it. • Timestamp can not be inserted into the “Sync” packet before it goes through the MACSec • Encryption must always take same number of cycles (even after key rollover); any jitter can impact PTP performance. • PHY will not be able to differentiate encrypted PTP packets with other packets • On receive side, decryption should take same time as encryption to avoid PDV from asymmetry #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 45 PTP over IPSec Deployment Challenges PTP over IPSec • IPSec is end-to-end and introduces additional delay. PTP • IPSec Engine could be 1) Hardware based or 2) Software based a. Hardware based IPSec Engine IPSec Engine PHY Egress Ingress • Challenges are similar to MACSec b. Software Based IPSec Engine • Time-stamping in software increases delay variations • Hardware based timestamping requires further considerations • Ingress packets has to be stored until corresponding PTP messages are decrypted and timestamped • Outgoing packets should follow 2-step method #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 46 Architectural Approach PTPoMACSec T-BC PTPoMACSec T-BC PTPoMACSec T-BC • MACSec performs best with Full Path Support (FPS) and ITU-T G.8275.1 profile • IPSec is end to end (Client to Server) and doesn’t align with G.8275.1 profile. T-BC PTPoMACSec PTP unaware network T-BC • 5G timing requirements can not be met when PTP over MACSec is deployed on Partial Timing Support (PTS) networks with ITU-T G.8275.2 PTPoIPSec T-BC T-BC T-BC • IPSec is end to end - doesn’t meet 5G performance requirement • Underlay network has no visibility to PTP packets #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 47 Security Mechanisms Guidelines IEEE 1588-2019 Annex P • Prong A: PTP integrated security using TLVs • Prong B: External Security Mechanisms • L2 with MACSec i.e. Hop by hop • L3 with IPSec i.e. End to End • Prong C: Architecture Guidance • Prong D: Monitoring and Management Guidance #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 48 • PTP Security Status • MACSec • MACSec with ClearTag is being discussed, however it doesn’t solve all issues. • Inter-op will be biggest challenge IPSec • • Not suitable to meet 5G RAN timing requirements Industry is aligning towards supporting • IEEE 1588-2019 prong A: PTP integrated security using TLV • Key Exchange and Processing mechanisms to be finalized • @ ITU-T - “Further Study” BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 49 Design Guidelines Network Design Phase Synchronization • • • • • • Deploy G.8275.1 with physical frequency (SyncE/eSyncE) Reduce PDV (boundary clocks reset PDV back to zero) Reduce asymmetry (routing, link, node, transport) - boundary clocks limit asymmetry Likely requires remediation of transport layer Avoid “PTP over Loopback” or ”PTP over un-numbered” interfaces Deploy G.8275.1 in transport with bundle interfaces, avoid G.8275.2 Frequency Synchronization • • • • If possible, use physical distribution (SyncE/eSyncE) not packet for frequency G.8265.1 interoperates with SONET/SDH & SyncE (some gaps with eESMC mapping) Packet distribution goal: reduce PDV (minimize hops as it disallows aware nodes) Asymmetry isn’t an issue for frequency over packet #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 51 Select the Correct Profile… Feature G.8275.1 PTPoE G.8275.2 PTPoIP Transport Layer 2 Multicast Layer 3 (IP) Unicast Network Model Full on-path support Partial on-path support IP Routing Not applicable Problematic (rings, asymmetry) Transit traffic Not allowed Problematic (jitter, asymmetry) Performance Best Variable Configuration Model Physical Port L3 device PTP over Bundles No issue Being worked on (for BC’s) Asymmetry Reduced (T-BC on every node) T-BC good, not T-BC = bad PDV/jitter Timestamping on wire (small) T-BC good, not T-BC = bad #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 52 Remediating Optical Transport • “Smart” optical devices introduce buffering and/or complex processing • Optical systems: separate channel for PTP (like Optical Service Channel) • Cisco solution based on the NCS2K * Bi-directional fiber reduces asymmetry T-BC T-BC PRTC / T-GM #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 53 Summary 1. 2. Key Take-a-ways Network based Time Synchronization is un-avoidable in 5G RAN • Reduce compute complexity and cost • Provides Resiliency and Redundancy Advanced features matures 5G Synchronization • Class C Clock, PRTC-B Receivers • eSyncE, eESMC • VP and APTS 3. PTP Security is not mature yet 4. Follow Design Best Practices for predictive timing performance. BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 55 Further Information “Synchronizing 5G Mobile Networks” • Publisher: Pearsons/Cisco Press • eBook & Print • Published: June 2021 https://www.ciscopress.com/store/ synchronizing-5g-mobilenetworks-9780136836254 BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 56 5G Timing & Synchronization • Resources: Past Cisco Live Sessions BRKSPG-3050 Synchronizing 5G Mobile Networks Event: 2023 Amsterdam • BRKSPM-3295 5G Timing & Synchronization architectures Event: 2020 Barcelona • BRKSPG-2557 5G Synchronization Design, Testing and Deploying Timing to support 5G rollouts Event: 2020 Barcelona BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 57 Recommended Sessions on 5G Cisco Private 5G Learning Map Start June 4 | 2:00 pm June 5 | 11:00 am June 7 | 4:00 pm New Adventures in Wireless: The Journey of WiFi6 and Private 5G Networks for the Enterprise Is your Enterprise Network Ready for P5G Monolithic or Polylithic packet cores? The case for specialized use-casebased mobile packet cores TECSPG-2432 BRKSPG-3004 BRKENS-2950 June 5, | 8:00 am June 5 | 11:30 am Architecting Enterprise Security in a Wi-Fi plus Private 5G World Leading Your Digital Transformation with Cisco Private 5G Network Offer BRKSEC-2085 June 5 | 8:30 am BRKSPG-2042 Architecting Private 5G for resiliency, security, and enterprise network convergence June 5 | 10:30 am BRKSPM-1006 The 5G System as a Spectrum Management Solution PSOSPG-1002 June 8 | 09:30 am BRKSPG-2044 5G Use Cases Flight Line of the Future and Smart Warehouse June 6 | 3:00 pm June 8 | 01:00 pm WiFi6 and Private 5G for the Enterprise – a ‘Better Together’ Journey Getting Started with Private 5G PSOGEN-1033 BRKGEN-2001 BRKEWN-2030 IBOSPG-2007 June 7 | 2:30 pm June 8 | 1:00 pm Unlock business outcomes from connectivity with a Private 5G solution Cisco P5G - A Robust and Secure Architecture Finish #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 59 Cisco 5G Learning Map Start June 4 | 9:00 am June 5 | 2:30 pm June 6 | 2:30 pm Designing IoT Wireless Networks Don’t Just Connect, Grow your IoT Business with Cisco IoT Cellular Connectivity Management 5G Transport Design Considerations Combining Onsite and Cloud-Based Deployments TECIOT-2584 June 5, | 8:30 am BRKNWT-2203 Automation-first Approach to Network Infrastructure Modernization for 5G & Beyond BRKSPG-1002 IBOSPM-2030 June 5 | 3:00 pm June 6 | 4:00 pm BRKIOT-1126 Connecting Moving Assets with Cisco IoT Solutions June 6 | 10:30 am Design, Deploy and Manage Transport Slicing using SDN Controller and Assurance Cloud-Ready Converged SDN Transport BRKSPG-2133 BRKSPG-2315 June 5 | 1:00 pm June 6 | 1:00 pm Hiking the Band Canyon with 5G: New Use Cases, New Business Outcomes Cisco Secure Edge Protection – Protecting the 5G Edge against DDoS Attacks BRKARC-2094 BRKSPG-2401 #CiscoLive BRKSPG-3050 Holistic Security in 5G Deployments June 7 | 10:30 am June 5 | 1:00 pm BRKSPG-2063 BRKSPM-2027 Evolution of the Transport Network Architecture in the Context of 5G and Open RAN June 8 | 8:30 am BRKSPG-3050 Synchronizing 5G Mobile Networks © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 60 Cisco 5G Learning Map June 8 | 9:30 am IBOSPG-2006 DISH Wireless, World’s first 5G Network with a Hybrid Cloud June 8 | 10:30 am BRKNWT-2301 DevNetOps Automation Approach to Network Infrastructure Modernization for 5G and Beyond June 8, | 11:00 am BRKSPG-2040 Troubleshooting 5G Architectures June 8 | 1:00 pm BRKMER-2001 Postcards from the 5G Edge: Meraki Cellular Gateways Finish #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 61 Reference: Inclusion and Diversity IF-1002 Learn how Inclusive Language Can Positively Impact your Business @ Panel Discussion DEVLIT-2761 Automating Language Bias out of Code BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 62 Fill out your session surveys! Attendees who fill out a minimum of four session surveys and the overall event survey will get Cisco Live-branded socks (while supplies last)! Attendees will also earn 100 points in the Cisco Live Game for every survey completed. These points help you get on the leaderboard and increase your chances of winning daily and grand prizes #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 63 Continue your education • Visit the Cisco Showcase for related demos • Book your one-on-one Meet the Engineer meeting • Attend the interactive education with DevNet, Capture the Flag, and Walk-in Labs • Visit the On-Demand Library for more sessions at www.CiscoLive.com/on-demand BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 64 Thank you #CiscoLive Gamify your Cisco Live experience! Get points for attending this session! How: 1 Open the Cisco Events App. 2 Click on 'Cisco Live Challenge’ in the side menu. 3 Click on View Your Badges at the top. 4 Click the + at the bottom of the screen and scan the QR code: #CiscoLive BRKSPG-3050 © 2023 Cisco and/or its affiliates. All rights reserved. Cisco Public 66 #CiscoLive