CISCO IOS IP SERVICE LEVEL AGREEMENTS: TECHNICAL OVERVIEW TOM ZINGALE INTERNET TECHNOLOGIES DIVISION SEPTEMBER 2004 Cisco IOS IP SLA, Technical, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Internal Use Only 1 Cisco IOS IP Service Level Agreement: A New Direction • Cisco solution that assures IP service levels, proactively verifies network operation, and accurately measures network performance Comprehensive hardware support Committed Cisco partner support Cisco IOS Software, the world’s leading network infrastructure software Enterprise and Small Medium Business Understand Network Performance & Ease Deployment Access Service Providers Verify Service Levels Verify Outsourced SLAs Enterprise Premise Edge Enterprise Backbone Measure and provide SLAs Service Provider Aggregation Edge Service Provider Core Cisco IOS Software Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 2 The Need for IP-Based Service Levels PROBLEM 40% of companies delay launching new applications due to network performance concerns2 RESULT Reduced business productivity 59% of companies simply add bandwidth to ensure application efficiency2 Increased network costs 55% of companies only identify some of their network traffic2 Reduced understanding of network behavior Cost of application downtime and degradation is $13,000 per minute for an ERP application3 Lowered network performance can be costly 2003 Infonetics Research Study “Cost of Enterprise Downtime” www.infonetics.com/services/green.shtml?2004/service.provider.and.user.plans.shtml 2 2003 Network World Application Performance Market Study www.nwfusion.com 3 Forrester Research Cisco IOSwww.forrester.com IP SLA, and 1 NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 3 Cisco IOS IP SLA Benefits OPTIMIZED APPLICATIONS & SERVICES • Performance visibility • Prove service levels • Enhance Customer satisfaction • Enhance acceptance of businesscritical services REDUCED TOTAL COST OF OWNERSHIP AND OpEx • Reduce deployment time • Lower mean time to restore and downtime • Proactive identification of issues enforces higher reliability Continuous Predictable Reliable Measurements and Metrics Automated Intelligence Proactive Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 4 Cisco IOS IP SLAs Life Cycle Baseline network performance Verify network readiness for new services with Cisco IOS IP SLA capabilities. 2 1 Quantify results • Reduce deployment time • Prove service and application differentiation • Verify service levels • Reduce network down time • Manage demand for the network Cisco IOS IP SLA, and NetFlow, 9/04 Understand network performance baseline Confidence to deploy new IP services and applications © 2004 Cisco Systems, Inc. All rights reserved. Assure application and service deployment 3 4 Cisco Confidential Fine tune and optimize Ongoing measurements to understand behavior with proactive notification 5 Example: Multi-Protocol Measurement and Management with Cisco IOS IP SLAs Applications Network Performance Monitoring Availability VoIP Monitoring Service Level Agreement (SLA) Monitoring Network Assessment Multiprotocol Label Switching (MPLS) Monitoring Trouble Shooting Measurement Metrics Packet Loss Latency Network Jitter Dist. of Stats Connectivity Protocols Jitter FTP DNS DHCP DLSW ICMP UDP TCP HTTP LDP H.323 SIP RTP Radius Video IP Server Defined Packet Size, Spacing COS and Protocol IP Server Cisco IOS Software Source IP SLA MIB Data Cisco IOS Software Active Generated Traffic to measure the network Destination Cisco IOS Software IP SLA IP SLA Responder Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 6 Comprehensive Hardware Support Core Enterprise & Aggregation/Edge Cisco IOS Software Release 12.2S Cisco 7200 Series Cisco Catalyst Cisco 10000 6500; Cisco Series 7600 Series Cisco 7300 Series Cisco 12000 Series Access Cisco IOS Software Releases 12.3T and 12.4 Cisco 800 Cisco 1700 1800 Series Series Cisco IOS IP SLA, and NetFlow, 9/04 Cisco 2600 2800 Series © 2004 Cisco Systems, Inc. All rights reserved. Cisco 7200 & Cisco 3700 7300 Series 3800Series Cisco Confidential Cisco 2900, 3550, & 3750 Series 7 SLA Verification and Management • Access router may be managed or unmanaged • Data typically provided by the service provider for the customer includes availability, QoS, and Jitter SLAs • Service Provider needs visibility in the Customer Edge, in order to commit to SLAs • Enterprise will verify SP SLAs by using access router edge to edge measurements Enterprise may provide restricted Simple Network Management Protocol (SNMP) (RTT, Latency, QoS) visibility into Access router for Service Provider Service Provider with restricted access can report SLA as a service back to the enterprise Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 8 Network Monitoring • Cisco IOS IP SLA answers the following question: What is the jitter, latency, or packet loss between any two points in the network? • IP Services can be simulated by specifying various packet sizes, ports, class of service, packet spacing, and measurement frequencies • Uni-directional and highly accurate measurements • Measurements per class of service to validate service differentiation for data, voice, and video • Cisco IOS IP SLA will identify an edge to edge network performance baseline and allow the user to understand trends and anomalies from the baseline Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 9 IP Network Readiness • Network assessment tool built into Cisco IOS Software • Simulate IP Services and verify how well they will work in the network • How well is QoS working in the network predeployment • Post deployment continued verification of network performance per IP service Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 10 Availability Monitoring • Cisco IOS IP SLA uses proactive monitoring for periodic, reliable, and continuous availability measurements • Connectivity measurements from Cisco router to router or Cisco router to server • Threshold notifications when end point is not available What is the availability of a Network File System (NFS) server used to store business critical data from a remote site ? Cisco IOS IP SLA UDP active measurement to specific server ports is used to test remote site to server connectivity If server is unavailable, then traps can notify the network management system Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 11 Troubleshooting with Cisco IOS IP SLA • Proactive notification of problems and issues based on threshold alerts • Testing edge to edge consistently and reliability will save time in finding and pin pointing network performance problem areas • Secondary activation of path operation (ie: path jitter) or activation of operations at a higher frequency to isolate and verify problem areas in the network Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 12 Cisco IOS IP SLA Source and Responder • Source Router Cisco IOS Software router that sends data from operation Cisco IOS Software may or may not be the target Some operations require the target to run the IP SLA responder Stores results in MIB • Responder Responds to IP SLA packets at destination User defined UDP/TCP ports IP SLA Control Protocol MD 5 Authentication Accurate measurements Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 13 Responder Source Router Target Router Responder T2 T1 T3 T4 D = T3 - T2 The Responder takes 2 Timestamps (T2 & T3) • Responder factors out destination processing time making results highly accurate • Responder allows for one-way measurements for latency, jitter, packet loss, and MOS Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 14 Cisco IOS IP SLAs Uses and Metrics *DATA TRAFFIC *VoIP REQUIREMENT • Minimize Delay, Packet Loss • Verify Quality of Service (QoS) IP SLA MEASURMENT • • • • Jitter Packet loss Latency per QoS • Minimize Delay, Packet Loss, Jitter • • • • *SERVICE LEVEL AGREEMENT • Measure Delay, Packet Loss, Jitter • One-way Jitter Packet loss Latency MOS Voice Quality Score • Jitter • Packet loss • Latency • One-way • Enhanced accuracy • NTP *AVAILABILITY Connectivity testing • Connectivity tests to IP devices **STREAMING VIDEO • Minimize Delay, Packet Loss • Jitter • Packet loss • Latency * Currently available **Limited availability in 9/04; complete in CY’05 Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 15 Cisco IOS IP SLA Reaction Conditions • Reaction Trigger to Events Can send SNMP traps for certain “triggering” events: Connection Loss and Timeout Round Trip Time Threshold Average Jitter Threshold Unidirectional packet loss, latency, jitter, MOS Scores Trigger • Immediate • Consecutive • X of Y times • Average Exceeded Can trigger another IP SLA operation for further analysis Threshold Violation Alert No Alert Threshold Violation Alert 100 ms 50 ms Threshold violation Time Cisco IOS IP SLA, and NetFlow, 9/04 Technical, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Resolution Cisco Cisco Internal Confidential Use Only 16 Availability 12.0(5)T 12.0(8)S 12.1E 12.1(1)T 12.2 12.2(2)T X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X UDP Jitter One Way Latency X FTP Get X Feature/Release 11.2 12.0(3)T ICMP Echo X X X X X X X X ICMP Echo Path SSCP(SNA) UDP Echo TCP Connect UDP Jitter HTTP DNS DHCP DLSw+ SNMP Support MPLS/VPN Aware Frame-Relay (CLI) ICMP Path Jitter APM Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. 12.2(11)T (Infra2) 12.2(14)S 12.2(25)S X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X X Cisco Confidential X X X X X X X X 17 Cisco IOS IP SLA Partners Cisco Network Management Solution Cisco IP Solution Center MPLS VPN and SLA Monitoring Internetworking Performance Monitor Enterprise performance measurements THIRD PARTY PRODUCTS Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 18 Cisco IOS IP SLA Performance with Infrastructure 2: CPU Load by Hardware • Jitter probe • Versus Release 12.3(3) Operations/ Second Operations/ Minute Cisco 2600 Series 4 8 12 16 240 480 720 960 14 20 29 35 20 24 28 32 36 40 44 1200 1440 1680 1920 2160 2400 2640 41 48 56 63 67 48 52 56 60 2880 3120 3360 3600 • Cisco 2620XM Series 7 8 12 15 2,000 active probes Cisco 3640 Series Cisco 3725 Router 6 9 13 17 2 3 2 3 Cisco 7200VXR NPE225 4 3 3 3 19 24 27 28 31 34 38 22 25 28 31 35 38 43 2 3 3 2 2 3 4 3 3 3 4 3 7 8 42 46 48 52 47 49 43 58 5 5 6 6 8 10 11 11 *Jitter operations are activated sequentially with this testing. Each operation sends 10 packets, 64 bytes each with 20ms spacing Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 19 Cisco IOS IP SLA Performance Infrastructure 2: CPU Load by Hardware • Jitter probe • Release 12.3(4)T6 • 2,000 active probes IP Plus/Firewall/3DES Operations per second 4 8 12 16 20 24 28 32 36 40 44 48 52 56 60 Cisco IOS IP SLA, and NetFlow, 9/04 Operations per minute 240 480 720 960 1200 1440 1680 1920 2160 2400 2640 2880 3120 3360 3600 © 2004 Cisco Systems, Inc. All rights reserved. Cisco 831 Router 7 13 23 29 33 35 41 47 52 57 62 66 72 76 81 Cisco 837 Router 10 16 23 30 34 36 41 46 50 56 62 65 68 71 75 Cisco Confidential Cisco 1751 Router 3 8 10 17 22 27 29 32 35 39 43 48 53 59 62 20 Cisco IOS IP SLA VoIP Measurements Q1CY’05 Data Center Gatekeeper Call Manager Cluster Registration Delay Discovery Delay Headquarters H323 or SIP Post Dial Delay Seattle LA San Jose Sales Office Sales Office Sales Office Cisco IOS IP SLA, and NetFlow, 9/04 Responder © 2004 Cisco Systems, Inc. All rights reserved. New York Cleveland Detroit Cisco Confidential Boston Sales Office Sales Office 21 Digital Signal Processor Based IP SLA Measurements (Q3CY’05) • VoIP Active (test call) measurements using Real-time Transport Protocol (RTP) streams • Voice quality scores and voice metrics from the Digital Signal Processor (DSP) Call Control VoIP Metrics RTP IP SLA DSP IP Server Responder RTP IP SLA Cisco IOS IP SLA, and NetFlow, 9/04 Cisco IOS IP SLA RTP Operation Data © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 22 New IOS IP SLA CLI • The new IOS IP SLA CLI releases Q1CY05 in 12.3(RLS6)T • Phase 1 changes include new syntax for commands and new show commands New show commands: “show ip sla statistics” and “ show ip sla statistics details” Older show commands will be deprecated over time and replaced with the new show commands The RTR keyword was changed to IP SLA Monitor in CLI The new syntax is used in the presentation. The old syntax before 12.3(pi6)T is shown in the Appendix OLD CLI Router (config)#rtr 1 Router (config-rtr)#type echo protocol ipIcmpEcho 1.1.1.1 Router (config)#rtr schedule 1 start-time now Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. New CLI Router (config)#ip sla monitor 1 Router (config-sla-monitor)#icmp-echo 1.1.1.1 Router (config)#ip sla monitor schedule 1 start-time now Cisco Confidential 23 New Cisco IOS IP SLA Show Commands Q1CY’05 • Jitter operation “show ip sla monitor statistics (details)” Router#sh ip sla monitor statistics 15 Round trip time (RTT) Index 15 Latest RTT: 1 ms Latest operation start time: *05:43:28.720 UTC Fri May 28 2004 Latest operation return code: OK RTT Values Number Of RTT: 10 RTT Min/Avg/Max: 1/1/1 ms Latency one-way time milliseconds Number of one-way Samples: 0 Source to Destination one way Latency Min/Avg/Max: 0/0/0 ms Desination to source one way Latency Min/Avg/Max: 0/0/0 ms Jitter time milliseconds Number of Jitter Samples: 9 Source to Destination Jitter Min/Avg/Max: 20/20/23 ms Destination to Source Jitter Min/Avg/Max: 0/0/0 ms Packet Loss Values Loss Source to Destination: 0 Loss Destination to Source: 0 Out Of Sequence: 0 Tail Drop: 0 Packet Late Arrival: 0 Number of successes: 1 Number of failures: 0 Operation time to live: 3567 sec Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 24 New Cisco IOS IP SLA Show Commands Q1CY’05 • Jitter operation “show ip sla monitor statistics details” Round trip time (RTT) Index 2004 Latest RTT: 1 ms Latest operation start time: *08:41:09.937 PST Wed Oct 6 2004 Latest operation return code: OK Over thresholds occurred: FALSE RTT Values Number Of RTT: 10 RTT Min/Avg/Max: 1/1/1 ms Latency one-way time: Number of Latency one-way Samples: 0 Source to Destination Latency one way Min/Avg/Max: 0/0/0 ms Destination to Source Latency one way Min/Avg/Max: 0/0/0 ms Source to Destination Latency one way Sum/Sum2: 0/0 Destination to Source Latency one way Sum/Sum2: 0/0 Jitter time: Number of Jitter Samples: 9 Source to Destination Jitter Min/Avg/Max: 0/0/0 ms Destination to Source Jitter Min/Avg/Max: 0/0/0 ms Source to destination positive jitter Min/Avg/Max: 0/0/0 ms Source to destination positive jitter Number/Sum/Sum2: 0/0/0 Source to destination negative jitter Min/Avg/Max: 0/0/0 ms Source to destination negative jitter Number/Sum/Sum2: 0/0/0 Destination to Source positive jitter Min/Avg/Max: 0/0/0 ms Destination to Source positive jitter Number/Sum/Sum2: 0/0/0 Destination to Source negative jitter Min/Avg/Max: 0/0/0 ms Destination to Source negative jitter Number/Sum/Sum2: 0/0/0 Interarrival jitterout: 0 Interarrival jitterin: 0 Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 25 Cisco IOS IP SLA Multiple Operations Scheduling (Release 12.3(8)T) • Schedule multiple operations in one command • Scalable and sequential activation of IP SLA operations If the frequency is not specified, the default frequency will be the same as that of the schedule period) Reduced load on the network Consistent monitoring coverage Router (config)#ip sla monitor 1 Router (config-sla-monitor)#type echo protocol ipIcmpEcho 1.1.1.1 Router (config)# ip sla monitor 2 Router (config-sla-monitor)#type echo protocol ipIcmpEcho 2.2.2.2 Router (config)# ip sla monitor 3 Router (config-sla-monitor)#type echo protocol ipIcmpEcho 3.3.3.3 Router (config)# ip sla monitor group schedule 1 1-3 sch 20 start now Router #show ip sla monitor group schedule Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 26 Cisco IOS IP SLA Random Scheduler Enhancement • Release 12.4(Rls1)T will introduce the following functionality: Randomness for group scheduler during schedule period Randomness for the frequency of the operations, which are started by random group scheduler Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 27 Cisco IOS IP SLA Accuracy Feature • High performance and high accuracy measurements • Precision to .1 ms from current 1ms • Improve Cisco IOS IP SLA accuracy under forwarding load and for dedicated routers • Release 12.3(RLS6)T will introduce this functionality in Q1CY’05 Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 28 Cisco IOS IP Service Level Agreement Roadmap Feature Release Target Date Release 12.3T Features MOS and ICPIF Scores 12.3(4)T November 2003 One way latency, jitter, packet loss and MOS Traps 12.3(7)T March 2003 Multi-Operation Scheduler – Ease of scheduling 12.3(8)T June 2003 Post Dial and Gatekeeper Delays with SIP and H323 12.3(pi-6)T Q1CY’05 High accuracy enhancement 12.3(pi-6)T Q1CY’05 Ease of use CLI 12.3(pi-6)T Q1CY’05 Release 12.4T Features Ease of use CLI Phase 2 12.4(pi-1)T Q2CY’05 Random scheduler for operations 12.4(pi-1)T Q2CY’05 Voice gateway integration VoIP measurement using DSP 12.4(pi-2)T Q3CY’05 Ease of use CLI Phase 3 12.4(pi-2)T Q3CY’05 Video operation 12.4(pi-2)T Q3CY’05 Radius response operation 12.4(pi-2)T Q3CY’05 Release 12.2S Features IP SLA: Auto MPLS VPN Monitoring 12.2(Rls6)S Q1CY’05 IP SLA: Auto MPLS VPN Monitoring with ECMP 12.2(Rls7)S Q3CY’05 IP SLA: Auto MPLS Monitoring with VCCV 12.2(Rls8)S Radar IP SLA: Auto MPLS Monitoring with BFD 12.2(Rls8)S Radar Radar IP SLA Multicast Radar Radar Auto IP SLA Monitoring Radar Radar IP SLA with DMVPN Radar Radar ICMP Jitter Radar Radar IP SLA High Availability Radar Radar Cisco IOS IP SLA, and NetFlow, 9/04 Embedded 2004 Cisco Systems, All rights reserved. Event ©Manager (EEM)Inc.Detector Radar Cisco Confidential Radar 29 NetFlow Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 30 Flow Is Defined By Seven Unique Keys • Source IP address • Destination IP address Traffic Enable NetFlow New SNMP MIB Interface • Source port • Destination port • Layer 3 protocol type • TOS byte (DSCP) • Input logical interface (ifIndex) NetFlow Export Packets Traditional Export & Collector SNMP Poller GUI Cisco Cisco IOS IOS IPNetFlow SLA, and NetFlow, Overview, 9/04 2/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 31 NetFlow Cache Example 1. Create and update flows in NetFlow cache Srclf SrclPadd Dstlf DstlPadd Protocol TOS Flgs Pkts Src Port Src Msk Src AS Dst Port Dst Msk Dst AS NextHop Bytes/ Pkt Active Idle Fa1/0 173.100.21.2 Fa0/0 10.0.227.12 11 80 10 11000 00A2 /24 5 00A2 /24 15 10.0.23.2 1528 1745 4 Fa1/0 173.100.3.2 Fa0/0 10.0.227.12 6 40 0 2491 15 /26 196 15 /24 15 10.0.23.2 740 41.5 1 Fa1/0 173.100.20.2 Fa0/0 10.0.227.12 11 80 10 10000 00A1 /24 180 00A1 /24 15 10.0.23.2 1428 1145.5 3 Fa1/0 173.100.6.2 Fa0/0 10.0.227.12 6 40 0 2210 19 /30 180 19 /24 15 10.0.23.2 1040 24.5 14 • • • • 2. Expiration Inactive timer expired (15 sec is default) Active timer expired (30 min (1800 sec) is default) NetFlow cache is full (oldest flows are expired) RST or FIN TCP Flag Srclf SrclPadd Dstlf DstlPadd Protocol TOS Flgs Pkts Src Port Src Msk Src AS Dst Port Dst Msk Dst AS NextHop Bytes/ Pkt Active Idle Fa1/0 173.100.21.2 Fa0/0 10.0.227.12 11 80 10 11000 00A2 /24 5 00A2 /24 15 10.0.23.2 1528 1800 4 3. Aggregation e.g. Protocol-Port Aggregation Scheme Becomes 4. Export version 5. Transport protocol Cisco IOS IP SLA, and NetFlow, 9/04 Export Packet © 2004 Cisco Systems, Inc. All rights reserved. Heade r Non-Aggregated Flows—Export Version 5 or 9 Payload (Flows) Protocol Pkts SrcPort DstPort Bytes/Pkt 11 11000 00A2 00A2 1528 Aggregated Flows—Export Version 8 or 9 Cisco Confidential 32 Principle Netflow Benefits Service Provider Enterprise • Traffic Engineering • Internet access monitoring (protocol distribution, where traffic is going/coming) • Accounting and billing • User Monitoring • Security Monitoring • Application Monitoring • Peering arrangements • Network Planning • Charge Back billing for departments • Security Monitoring Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 33 Tracking Users Who are the top users? How long are the users on the network? What Internet sites do they use? Where do the users go on the network? What percentage of traffic do they use? What applications do they use? What are the user usage patterns? Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 34 NetFlow for Security: Flow Information Helps Mitigate Attacks • Identify the attack Count the Flows Inactive flows signal a worm attack • Classify the attack Small size flows to same destination What is being attacked and origination of attack • Key Partners: Arbor Networks, Protego, NetQos, Adlex Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 35 Capacity Planning • Capacity planning is the process of determining the network resources required to prevent a performance or availability impact on business-critical applications • Key areas to monitor Application usage Identify which applications consume bandwidth Who are the top ten nodes that consume bandwidth • Output data circuit forecasts • Current network utilization and capacity being used Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 36 Billing • IP Accounting and Billing • Usage-based billing considerations Time of day Within or outside of the network Application Distance-based Quality of Service (QoS) / Class of Service (CoS) Bandwidth usage Transit or peer Data transferred Traffic class Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 37 How Cisco IT uses NetFlow • Characterize IP traffic and account for how and where it flows Total Avoidance of SQL Slammer Worm Transition from Managed DSL service to Internet VPN Detection of Unauthorized WAN Traffic Reduction in Peak WAN Traffic Validation of QoS Parameters and BW allocation Analysis of VPN Traffic and Tele-Commuter Behavior Calculating Total Cost of Ownership for Applications Use of NetFlow NMS and Usage Security Monitoring Network traffic analysis by application with BGP. Anomaly detection Arbor Networks WAN Aggregation and Edge Network traffic analysis by application, for capacity planning using NetQOS Core routers and Nat Gateway Collection of historical data, useful for forensics and diagnostics with Flow Tools Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 38 Comprehensive Hardware Support Enterprise & Aggregation/Edge Core Cisco IOS Software Release 12.2S Cisco 7200 Series Cisco 7300 Series Cisco Catalyst Cisco 10000 6500; Cisco Series ASIC 7600 Series Cisco 4500 Series ASIC Release 12.0S Cisco 12000 Series ASIC Access Cisco IOS Software Releases 12.3T & 12.4 Cisco 800 Cisco 1700 Series Series Cisco IOS IP SLA, and NetFlow, 9/04 Cisco 2600 Series © 2004 Cisco Systems, Inc. All rights reserved. Cisco 3700 Series Cisco 7200/ 7300 Series Cisco Confidential 39 NetFlow Versions NetFlow Version 1 5 7 8 9 Comments Original Standard and most common Specific to Cisco Catalyst 6500 and 7600 Series Switches Similar to Version 5, but does not include AS, interface, TCP Flag & TOS information Choice of eleven aggregation schemes Reduces resource usage Flexible, extensible file export format to enable easier support of additional fields & technologies; coming out now MPLS, Multicast, & BGP Next Hop Cisco Catalyst 6500 Series Router will support versions 5 & 8 in Cisco IOS Software Release 12.1(13)E Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 40 Version 5 - Flow Export Format Usage • Packet Count • Byte Count Source IPIPAddress •• Source Address Destination IPIP Address •• Destination Address From/To Time of Day • Start sysUpTime • End sysUpTime • Source TCP/UDP Port • Destination TCP/UDP Port Application Port Utilization • Input ifIndex • Output ifIndex QoS • Type of Service • TCP Flags • Protocol • Next Hop Address • Source AS Number • Dest. AS Number • Source Prefix Mask • Dest. Prefix Mask Routing and Peering Version 5 used extensively today Flow information Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 41 Why a New Version 9? • Fixed export formats are not flexible and adaptable • With each new version Cisco creates new export fields • Partners need to re-engineer for each new version Solution: Build a flexible and extensible export format called version 9! Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 42 NetFlow v9 Export Packet To support technologies such as MPLS or Multicast, this export format can be leveraged to easily insert new fields Template FlowSet (version, # packets, sequence #, Source ID) Template Record Template ID #1 Template Record Template ID #2 (specific Field types and lengths) (specific Field types and lengths) Flows from Interface A Flows from Interface B Data FlowSet Data FlowSet FlowSet ID #1 FlowSet ID #2 Data Record (Field values) Option Template FlowSet Data Record Data Record Template ID (Field values) (Field values) (specific Field types and lengths) Option Data FlowSet FlowSet ID Option Data Record Option Data Record (Field values) (Field values) • Matching ID numbers are the way to associate template to the Data Records • The Header follows the same format as prior NetFlow versions so Collectors will be backward compatible • Each data record represents one flow • If exported flows have the same fields, then they can be contained in the same Template Record (ie: unicast traffic) can be combined with multicast records • If exported flows have different fields, then they cannot be contained in the same Template Record (ie: BGP next-hop cannot be combined with MPLS Aware NetFlow Cisco IOSrecords) IP SLA, and Cisco Confidential NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. 43 NetFlow v9 and IETF • Internet Protocol Flow Information eXport (IPFIX) is an IETF Working Group www.ietf.org/html.charters/ipfix-charter.html • Netflow version 9 is the basis for the standard in the IETF • Standards Track NetFlow version 9 http://www.ietf.org/internet-drafts/draft-ietf-ipfix-protocol-05.txt Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 44 IETF: Packet SAMPling WG (PSAMP) • PSAMP web site for the charter, email archive, drafts, etc. psamp.ccrle.nec.de/ • Agreed to use IPFIX for export protocol if suitable for PSAMP To be improved: the variable length data type • Note: NetFlow is already using some sampling mechanisms Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 45 NetFlow Partners Traffic Analysis Flow-Tools Denial of Service Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Billing Cisco Confidential 46 Cisco Catalyst 6500 Series Switch and Cisco 7600 Series Router • Hybrid: Cisco Catalyst OS on PFC/supervisor and Cisco IOS software on MSFC • Native Cisco IOS Software: PFC/supervisor and the MSFC both run a single bundled Cisco IOS software image • Export is centrally via the supervisor and MSFC, each linecard has its own hardware NetFlow cache and forwarding table, i.e. distributed platform Hybrid Native 12.1E Native 12.2SX MSFCx v5 v5 v5, v8* Sup1a V7, v8 v7 N/A Sup2 V7, v8 v5, v7 v5, v7, v8 v5, v7, v8 v5, v7 v5, v7, v8 Sup720 *No NetFlow Support on MSFC with Sup1a Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 47 Cisco Catalyst 6500 and Cisco 7600 Series Versions and Features • Cisco IOS Software Release 12.1(13)E1 PFC2 Source/destination interface information (Hybrid 6.3(6)) PFC2 Source/destination AS information PFC2 Support for V5 NetFlow data export (Hybrid 7.5(1)) IP Next hop Sampled NetFlow is available on PFC in Cisco IOS • Cisco IOS Software Release 12.2(14)SX Version 8 in native mode • PFC3b (Sup720) cards ToS byte • Hybrid Catalyst OS 7.2(1) L2 switched traffic (vlan x to vlan y) support (doesn’t require MSFC) • Hybrid Catalyst OS 7.3(1) Destination and source IfIndex enabled by default Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 48 Cisco Catalyst 4000 Supervisor IV NetFlow Services Card NetFlow Service Card Features • NetFlow Statistics Collection and Data Export (NDE) •VLAN Statistics Collection •CLI support for NetFlow & VLAN Stats •SNMP support for VLAN Stats Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. • Requirements: • Supervisor IV or V • IOS 12.1(13)EW • NetFlow Versions 1 & 5, 8 w IOS 12.1.19 EW Cisco Confidential 49 NetFlow Features supported with Version 9 • Multicast NetFlow Availability: Major Release 12.3(1) and 12.2(18)S Ingress Accounting of replicated multicast packets Egress Per user accounting of multicast packets • MPLS Aware NetFlow Availability: Release 12.0(26)S Label and prefix export information • BGP Next Hop Availability: Releases 12.0(26)S, 12.2(18)S, and 12.3 Edge to Edge Traffic Matrix BGP traffic destination information • NetFlow for IPv6 Availability: Release 12.3(7)T Export IPv6 source and destination information Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 50 NetFlow Product Update • Sampled NetFlow Availability: Releases 12.0(26)S, 12.3(2)T, and 12.2(18)S Random Sampling of packets per flow with reduce CPU • NetFlow MIB Availability: Releases 12.3(7)T and 12.2(25)S Top N Talker in MIB NetFlow configuration using MIB • Input Flow Filters Availability: Release 12.3(7)T, 12.2(25)S QOS MQC based Filtering entering NetFlow • Egress NetFlow Availability: Release 12.3(11)T, 12.2(Rls6)S-Q1CY05 Accounting for Egress IP Flows Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 51 Random Sampled NetFlow • Capacity planning may not need every packet per Flow • Sampling on high speed interfaces will reduce CPU consumption • Random (select packet to export per statistical principles) Cisco IOS Software Releases 12.0(26)S, 12.2S(18), and 12.3(1)T Cisco 800, 1700, 1800, 2600, 2800,3600, 3700, 3800 7200, and 7500 Series Routers Random sampling Cisco 12000 Series 12.0(28)S Cisco 12000 Series deterministic sampling today Cisco Catalyst 6500 Series Random and Time based sampling 12.1(13)E Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 52 NetFlow MIB • Currently available in Releases 12.3(7)T • NetFlow information available using SNMP and without NetFlow export • Administration of Netflow using the MIB interface • NetFlow MIB cannot be used to retrieve all Flow information but is very useful for security monitoring and locations where export is not possible Example objects available: Packet size distribution Number of Bytes exported per second Number of flowsNetFlow MIB with Export of Top N talkers • Top N Talkers Top N Flows based on various NetFlow field values ( AS Number, destination, ports…) MIB and CLI support 12.2(25)S and 12.3(11)T Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 53 Import Flow Mask Filters • Prevent flows from entering NetFlow cache by using Flow Filter • Increase scalability and decrease CPU usage • Filters are based on QOS MQC CLI class maps • User can use ACL to match flows from certain port or source • Define Traffic Class (match ACL) and Flow Sampling per Match 12.0(27)S, 12.3(4)T, 12.2S(25) Traffic Filter High Importance Sample 1:1 from Server B Packets Traffic Filter Low Importance Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Sample 1:100 from Subnet A Cisco Confidential 54 Egress NetFlow Accounting Netflow Egress and Ingress PE PE IP Servers IP IP or MPLS Netflow Ingress Netflow Egress 12.3(7)T, 12.2(25)S Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 55 Flexible NetFlow and Flexible Accounting • Flexible NetFlow and Flexible Accounting will replace most static accounting technologies available today Flexible NetFlow user defined Flow keys and export fields within NetFlow Flexible Accounting user defined permanent flow with periodic export and account for defined flows over time The data can be polled thru a MIB Flow Groups user defined buckets for specific flow fields values Example show me packets and bytes from 1.1.1.1 to 2.2.2.2 on port 21 Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 56 SCTP Reliable Transport • Flows may be sent in Reliable or unreliable or partial mode • SCTP connection to collector and multiple streams per connection • Supported with Version 9. Templates may be sent reliably • Congestion Awareness, retransmission and queuing Send Queue Releases 12.4(2nd)T, 12.2S(Rls7) Data for Export in SCTP Stream Cisco IOS IP SLA, and NetFlow, 9/04 Congestion packets marked unreliable potentially dropped © 2004 Cisco Systems, Inc. All rights reserved. Collector Cisco Confidential 57 NetFlow Security Enhancement Releases 12.4(1st)T Q2CY05 • New show commands to understand and parse NetFlow data For Example, show flows on port X to destination Y show ip flow top <N> <aggregate-field> <sort-criteria> <matchcriteria> show ip flow top 10 destination-address packets interface ser0 port-range 100 to 135 • New Flow export fields including Source Mac, TTL, Packet length, ICMP type, and more • Also will be available in 12.2(rls7)S Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 58 Upcoming New Features: NetFlow Product Update • NetFlow Security Enhancements (Q2CY2005) New exports and show commands for security monitoring • Flexible NetFlow and Accounting (Q3CY2005) Allow user defined flow keys and aggregation with v.9 • Reliable and Congestion Aware Export (Q2CY2005) SCTP protocol NetFlow export • NBAR and NetFlow Integration (Radar) Application flow information export Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 59 NetFlow Roadmap Enhancing Cisco technologies’ with Flow Accounting Scalability & Flexibility Nov 2003 Dec 2003 Jan 2004 Feb 2004 Mar 2004 Apr 2004 May 2004 Jun 2004 Optimizing data for Flow processing Jul 2004 Aug 2004 Sep 2004 Oct 2004 • Input Filter • NetFlow MIB & Top Talker Targeting 12.3(11)T • NetFlow IPv6 Cisco IOS IP SLA, and NetFlow, 9/04 Dec 2004 Jan 2005 Feb 2005 Mar 2005 • Egress NetFlow Targeting 12.3(2)T • Input Filter Nov 2004 Targeting 12.2(Rls6)S 12.0(27)S 12.3(Rls2)T Standardization • Egress NetFlow • Flexible Flow Definition Reliable Export Targeting 12.2(25)S • NetFlow MIB & Top Talker Targeting 12.4(Rls1)T • Input Filter Security Exports © 2004 Cisco Systems, Inc. All rights reserved. Targeting 12.2(Rls7)S Cisco Confidential Security Exports MIB Phase 2 60 Cisco IOS IP SLA, and NetFlow, 9/04 © 2004 Cisco Systems, Inc. All rights reserved. Cisco Confidential 61