Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II Americas Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 527-0883 Cisco Validated Design The Cisco Validated Design Program consists of systems and solutions designed, tested, and documented to facilitate faster, more reliable, and more predictable customer deployments. For more information visit www.cisco.com/go/validateddesigns. ALL DESIGNS, SPECIFICATIONS, STATEMENTS, INFORMATION, AND RECOMMENDATIONS (COLLECTIVELY, "DESIGNS") IN THIS MANUAL ARE PRESENTED "AS IS," WITH ALL FAULTS. CISCO AND ITS SUPPLIERS DISCLAIM ALL WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE WARRANTY OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE. IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THE DESIGNS, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. THE DESIGNS ARE SUBJECT TO CHANGE WITHOUT NOTICE. USERS ARE SOLELY RESPONSIBLE FOR THEIR APPLICATION OF THE DESIGNS. THE DESIGNS DO NOT CONSTITUTE THE TECHNICAL OR OTHER PROFESSIONAL ADVICE OF CISCO, ITS SUPPLIERS OR PARTNERS. USERS SHOULD CONSULT THEIR OWN TECHNICAL ADVISORS BEFORE IMPLEMENTING THE DESIGNS. RESULTS MAY VARY DEPENDING ON FACTORS NOT TESTED BY CISCO. CCVP, the Cisco Logo, and the Cisco Square Bridge logo are trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and Learn is a service mark of Cisco Systems, Inc.; and Access Registrar, Aironet, BPX, Catalyst, CCDA, CCDP, CCIE, CCIP, CCNA, CCNP, CCSP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Cisco Unity, Enterprise/Solver, EtherChannel, EtherFast, EtherSwitch, Fast Step, Follow Me Browsing, FormShare, GigaDrive, GigaStack, HomeLink, Internet Quotient, IOS, iPhone, IP/TV, iQ Expertise, the iQ logo, iQ Net Readiness Scorecard, iQuick Study, LightStream, Linksys, MeetingPlace, MGX, Networking Academy, Network Registrar, Packet, PIX, ProConnect, RateMUX, ScriptShare, SlideCast, SMARTnet, StackWise, The Fastest Way to Increase Your Internet Quotient, and TransPath are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the United States and certain other countries. All other trademarks mentioned in this document or Website are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0612R) Any Internet Protocol (IP) addresses used in this document are not intended to be actual addresses. Any examples, command display output, and figures included in the document are shown for illustrative purposes only. Any use of actual IP addresses in illustrative content is unintentional and coincidental. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II © 2007 Cisco Systems, Inc. All rights reserved. Preface Revised: October 11, 2007 The aim of this document is to accelerate customer deployments of the Next Generation Enterprise MPLS-based WAN solution. It presents results and recommendations for all the deployment architectures outlined in the Next Generation Enterprise MPLS-Based WAN Design and Implementation Guide. Table 1 Modification History Date Comment Aug 2007 Initial Release Definitions This section defines words, acronyms, and actions which may not be readily understood. Table 2 Definitions Term Definition NSITE Network Systems Integration and Test Engineering CVD Cisco Validated Design CCM Cisco Unified Communications Manager VPN Virtual Private Network: A secure IP-based network that shares resources on one or more physical networks. A VPN contains geographically dispersed sites that can communicate securely over a shared backbone. DMVPN Dynamic Multipoint VPN MPLS Inter-AS Label Switching VRF VPN routing/forwarding instance. A VRF consists of an IP routing table, a derived forwarding table, a set of interfaces that use the forwarding table and a set of rules and routing protocols that determine what goes into the forwarding table. In general, a VRF includes the routing information that defines a customer VPN site that is attached to a PE router. WAN Wide Area Network Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 3 Preface Table 2 Definitions Term Definition MAN Metropolitan Area Network PE Provider Edge router: A router that is part of a service provider's network connected to a customer edge (CE) router. All VPN processing occurs in the PE router CE Customer Edge router: A router that is part of a customer network and that interfaces to a Provider Edge (PE) router. CE routers are not aware of associated VPNs. AS Autonomous System BGP Border Gateway Protocol: Interdomain routing protocol that exchanges reachability information with other BGP systems. It is defined in RFC 1163. SPA Shared Port Adapters SIP SPA Interface Processor NHRP Next Hop Resolution Protocol LDP Label Distribution Protocol Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 4 CONTENTS 1 CHAPTER 1 Cisco Validated Design Program 1-1 1.1 Cisco Validated Design I 1-1 1.2 Cisco Validated Design II 1-1 CHAPTER 2 Executive Summary CHAPTER 3 WAN Edge: MPLSoL2 Service 2-1 3-1 3.1 MPLSoL2 Test Coverage 3-1 3.1.1 MPLSoL2 Feature Coverage 3-2 3.1.2 CVD II Additional Coverage 3-3 3.2 MPLSoL2 Test Strategy 3-3 3.2.1 MPLSoL2 Test Topology 3-3 3.2.2 Test Types 3-4 3.2.2.1 System Integration Test 3.2.2.2 Scalability Test 3-5 3.2.2.3 Negative Test 3-5 3.2.2.4 Reliability Test 3-5 3.2.3 Sustaining Coverage 3-5 CHAPTER 4 3-5 3.3 MPLSoL2 Hardware and Software Information 3-6 3.4 MPLSoL2 Test Results and Recommendations 3.4.1 MPLSoL2 Test Results 3-7 3.4.2 MPLSoL2 Recommendations 3-8 3-7 WAN Edge: DMVPN per VRF 4-1 4.1 DMVPN per VRF Test Coverage 4-1 4.1.1 DMVPN per VRF Feature Coverage 4.1.2 CVD II Additional Coverage 4-3 4-2 4.2 DMVPN per VRF Test Strategy 4-3 4.2.1 DMVPN per VRF Test Topology 4-3 4.2.2 Test Types 4-4 4.2.2.1 System Integration Test 4-5 4.2.2.2 Scalability Test 4-5 Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 1 Contents 4.2.2.3 Negative Test 4-5 4.2.2.4 Reliability Test 4-6 4.2.3 Sustaining Coverage 4-6 CHAPTER 5 4.3 DMVPN per VRF Hardware and Software Information 4-6 4.4 DMVPN per VRF Test Results and Recommendations 4.4.1 DMVPN per VRF Test Results 4-7 4.4.2 DMVPN per VRF Recommendations 4-8 4-7 WAN Edge: MPLS over DMVPN - 2547oDMVPN (Hub and Spoke Only) 5.1 2547oDMVPN Test Coverage 5-1 5.1.1 2547oDMVPN Feature Coverage 5-2 5.1.2 CVD II Additional Coverage 5-3 5.2 2547oDMVPN Test Strategy 5-3 5.2.1 2547oDMVPN Test Topology 5-3 5.2.2 Test Types 5-4 5.2.2.1 System Integration Test 5-5 5.2.2.2 Scalability Test 5-5 5.2.2.3 Negative Test 5-5 5.2.2.4 Reliability Test 5-6 5.2.3 Sustaining Coverage 5-6 CHAPTER 6 5.3 2547oDMVPN Hardware and Software Information 5-6 5.4 2547oDMVPN Test Results and Recommendations 5.4.1 2547oDMVPN Test Results 5-7 5.4.2 2547oDMVPN Recommendations 5-8 5-7 WAN Core: Inter-AS, ASBR-to-ASBR with MPeBGP 6.1 Inter-AS Test Coverage 6-1 6.1.1 Inter-AS Feature Coverage 6-2 6.1.2 CVD II Additional Coverage 6-2 6.2 Inter-AS Test Strategy 6-3 6.2.1 Inter-AS Test Topology 6-3 6.2.2 Test Types 6-5 6.2.2.1 System Integration Test 6.2.2.2 Negative Test 6-5 6.2.3 Sustaining Coverage 6-5 6-5 6.3 Inter-AS Hardware and Software Information 6-6 6.4 Inter-AS Test Results and Recommendations 6.4.1 Inter-AS Test Results 6-7 6-7 Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 2 6-1 5-1 Contents 6.4.2 Inter-AS Recommendations CHAPTER 7 References 6-7 7-1 Test Coverage Matrix A-1 A.1 NG WAN Test Coverages Matrix Test Case Descriptions and Results B.1 MPLSoL2 Deployment Model A-1 B-1 B-1 B.2 DMVPNperVRF Deployment Model B-4 B.3 2547oDMVPN (Hub as PE Role) Deployment Model B.4 2547oDMVPN (Hub as P Role) Deployment Model B.5 Inter-AS Deployment Model Defects B-7 B-10 B-14 C-1 C.1 CSCsi44003 C-1 C.2 CSCsj78913 C-1 C.3 CSCek74416 C-2 C.4 CSCsi50615 C-2 C.5 CSCsi49487 C-2 C.6 CSCsi79767 C-3 Definition of Test Types D-1 Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 3 Contents Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 4 F I G U R E S Figure 3-1 MPLSoL2 Service Figure 3-2 MPLSoL2 Test Bed Figure 4-1 DMVPN per VRF Figure 4-2 DMVPN per VRF Test Bed Figure 5-1 2547oDMVPN (Hub as a P router) Figure 5-2 2547oDMVPN Testbed Figure 6-1 Inter-AS (ASBR-to-ASBR with MPeBGP) Figure 6-2 Inter-AS (ASBR-to-ASBR with MPeBGP) Testbed 3-2 3-4 4-2 4-4 5-2 5-4 6-2 6-4 Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 1 Figures Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 2 T A B L E S Table 1 Modification History Table 2 Definitions Table 2-1 WAN Edge Certification and Validation Summary 2-1 Table 2-2 WAN Core Certification and Validation Summary 2-2 Table 3-1 MPLSoL2 Hardware and Software Information Table 3-2 MPLSoL2 Test Results Summary Table 4-1 DMVPN per VRF Hardware and Software Information Table 4-2 TDMVPN per VRF Test Results Summary Table 5-1 2547oDMVPN Hardware Platforms and Data Table 5-2 25470DMVPN Test Results Summary Table 6-1 Inter-AS Hardware and Software Information Table 6-2 Inter-AS Test Results Summary Table A-1 MPLSoL2 Features Table A-2 MPLSoL2 CVDI Platforms and Software A-1 Table A-3 MPLSoL2 CVDII Platforms and Software A-2 Table A-4 DMVPNperVRF Features Table A-5 DMVPNperVRF CVDI Platforms and Software A-2 Table A-6 DMVPNperVRF CVDII Platforms and Software A-3 Table A-7 2547oDMVPN (Hub as PE Role) Features Table A-8 2547oDMVPN (Hub as PE Role) CVDI Platforms and Software A-3 Table A-9 2547oDMVPN (Hub as PE Role) CVDII Platforms and Software A-4 Table A-10 2547oDMVPN (Hub as P Role) Features Table A-11 2547oDMVPN (Hub as P Role) CVDII Platforms and Software Table A-12 Inter-AS (MAN CORE connection)Features Table A-13 Inter-AS (MAN CORE Connection) CVDII Platforms and Software Table B-1 MPLSoL2 Deployment Model Table B-2 DMVPNperVRF Deployment Model Table B-3 2547oDMVPN (Hub as PE Role) Deployment Model Table B-4 2547oDMVPN (Hub as P Role) Deployment Model Table B-5 Inter-AS Deployment Model 1-3 1-3 3-6 3-7 4-6 4-8 5-6 5-7 6-6 6-7 A-1 A-2 A-3 A-4 A-4 A-5 A-5 B-1 B-4 B-7 B-10 B-14 Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 1 Tables Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 2 CH A P T E R 1 Cisco Validated Design Program Revised: October 22, 2007 1.1 Cisco Validated Design I The Cisco® Validated Design Program (CVD) consists of systems and solutions that are designed, tested, and documented to facilitate faster, more reliable and more predictable customer deployments. These designs incorporate a wide range of technologies and products into a broad portfolio of solutions that meet the needs of our customers. There are two levels of designs in the program: Cisco Validated Design I and Cisco Validated Design II. Cisco Validated Design I are systems or solutions that have been validated through architectural review and proof-of concept testing in a Cisco lab. Cisco Validated Design I provide guidance for the deployment of new technology or in applying enhancements to existing infrastructure. 1.2 Cisco Validated Design II The Cisco Validated Design II (CVD II) is a program that identifies systems that have undergone architectural and customer relevant testing. Designs at this level have met the requirements of a CVD Validated design as well as being certified to a baseline level of quality that is maintained through ongoing testing and automated regression for a common design and configuration. Certified designs are architectural best practices that have been reviewed and updated with appropriate customer feedback and can be used in pre- and post-sales opportunities. Certified designs are supported with forward looking CVD roadmaps and system test programs that provide a mechanism to promote new technology and design adoption. CVD II Certified Designs advance Cisco System's competitive edge and maximize our customers' return on investment while ensuring operational impact is minimized. A CVD II certified design is a highly validated and customer representative solution that meets the following criteria: • Reviewed and updated for general deployment • Achieves the highest levels of consistency and coverage within the Cisco Validated Design program • Solution requirements successfully tested and documented with evidence to function as detailed within a specific design in a scaled, customer representative environment • Zero observable operation impacting defects within the given test parameters , that is, no defects that have not been resolved either outright or through software change, redesign, or workaround Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 1-1 Chapter 1 Cisco Validated Design Program 1.2 Cisco Validated Design II • A detailed record of the testing conducted is generally available to customers and field teams, which provides: – Design baseline that provides a foundational list of test coverage to accelerate a customer deployment – Software baseline recommendations that are supported by successful testing completion and product roadmap alignment – Detailed record of the associated test activity that includes configurations, traffic profiles, memory and CPU profiling, and expected results as compared to actual testing results For more information about the Cisco CVD program, refer to: http://www.cisco.com/go/cvd CVD II testing for this program was conducted by Cisco's Network System Integration and Test Engineering NSITE team. NSITE’s mission is to system test complex solutions spanning multiple technologies and products to accelerate successful customer deployments and new technology adoption. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 1-2 CH A P T E R 2 Executive Summary Revised: October 23, 2007 This document describes the CVD II validation of the Next Generation Enterprise MPLS-Based WAN Design and Implementation Guide. The aim of this project is to accelerate customer deployments of the Next Generation Enterprise MPLS-based WAN solution. Extensive manual and automated testing was conducted in a large scale, comprehensive customer representative network. The deployment architectures were validated with a wide range of system test types, including system integration, negative (fault and error handling), redundancy (availability), scalability and reliability to ensure successful customer deployment of the NG Enterprise MPLS-based WAN design. An important part of the testing is end-to-end verification of enterprise voice, and video services using components of the Cisco Unified Communications solution. Critical service parameters such as packet loss, end-to-end delay and jitter for voice and video were verified under load conditions. As an integral part of the CVDII program, an automated sustaining validation model was created for on-going validation of deployment architectures for future Internetworking Operating System (IOS) releases. With this automated sustaining validation capability, the sustaining team can validate the design in any upcoming software releases on the targeted platforms. Sustaining validation greatly extends the useful life of the design guide, and significantly increases customer confidence and reduces deployment time. During testing, there were a number of software defects encountered. The symptoms, conditions and workarounds of each defect are described Appendix C. Table 2-1, outlines the summary of certification and validation status of each of the WAN Edge deployment architectures. Table 2-2, outlines the summary of certification and validation status of each of the WAN Core deployment architectures. Table 2-1 WAN Edge Certification and Validation Summary MPLS-Based WAN Edge Deployment Architecture Status Status MPLSoL2 Passed with Exception1 Self-Deployed Multi-VRF with mGRE/DMVPN (DMVPN per VRF) Passed with Exception1 Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 2-1 Chapter 2 Table 2-1 Executive Summary WAN Edge Certification and Validation Summary MPLS-Based WAN Edge Deployment Architecture Status Status MPLS VPN over DMVPN - 2547oDMVPN (Hub Not Recommended and Spoke only, Hub as PE) (Use Hub as P role instead) MPLS VPN over DMVPN - 2547oDMVPN (Hub Passed with Exception as P Role) Table 2-2 WAN Core Certification and Validation Summary WAN Core (Interconnecting MPLS MANs) Status Inter-AS (ASBR-to-ASBR with MPeBGP) Passed 1 Exceptions to the CVD certification criteria were observed, however these are such that the design is still deployable for the majority of cases within the caveats defined for the exceptions which may only impact minor elements of the design or certain deployment scenarios. This document is intended to supplement to the CVD I, Next Generation Enterprise MPLS-Based WAN Design and Implementation Guide. A brief overview of each deployment model is presented from the CVD I document to provide background information. The CVD II test coverage, strategy, results and recommendations for each deployment model is presented independently within each chapter. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 2-2 CH A P T E R 3 WAN Edge: MPLSoL2 Service Revised: October 23, 2007 The MPLSoL2 deployment model is one of the possible solutions for branch virtualizations described in the Next Generation Enterprise MPLS-Based WAN Design and Implementation Guide (CVD I). A brief overview of the solution, test coverage, test strategy and a summary of the test results with recommendations are presented in this section. 3.1 MPLSoL2 Test Coverage The MPLSoL2 Service model assumes that the enterprise has existing Layer2 services for connecting branches and wants to enable MPLS over them. Since Layer2 connectivity is typically hub and spoke or partial mesh, the MPLS overlay also inherits the same connectivity characteristics. The branch aggregation router is converted into a P role for the MPLS network and is expected to label switch packets as shown in Figure 3-1 on page 3-2. The branch routers become PE routers with VRF interfaces facing the branch and MPLS-enabled interface facing the headend. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 3-1 Chapter 3 WAN Edge: MPLSoL2 Service 3.1.1 MPLSoL2 Feature Coverage Figure 3-1 MPLSoL2 Service MPLS MAN RR RR EP MPLS Enabled Links SP L2 Service E-PE Remote Branches E-PE PE 185835 E-PE 3.1.1 MPLSoL2 Feature Coverage The feature set in this deployment model closely follows what is recommended in the CVD I document. The testbed configuration assumes a redundant hub scenario. The following key features were tested; • MPLS VPN • MPLS Label Distribution Protocol (LDP) • MPBGP (Multiprotocol BGP) • Multicast VPN (mVPN) • OSPF • QoS: LLQ, CBWFQ, MPLS QoS, WRED • Source Specific Multicast (SSM) • Multicast over NBMA • Redundancy QoS The existing WAN QoS recommendation that were made in the Enterprise QoS Solutions Reference Network Design (SRND) still apply to MPLS WAN setup. An OC3 POS link was used at the headend so link efficiency policies such as LFI and cRTP were not needed. An 8-class QoS model was used at the WAN Edge with bandwidth allocation per the recommendations from the Enterprise QoS SRND. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 3-2 Chapter 3 WAN Edge: MPLSoL2 Service 3.1.2 CVD II Additional Coverage The branch routers had T1 links or higher so they used the same 8-class model with LLQ for voice and video, CBWFQ and WRED for all other classes. QoS testing involved generating enough traffic to congest the branch links as well as the headend link. A traffic generator was used to send various traffic types that match each QoS class. 3.1.2 CVD II Additional Coverage The C7200 with the NPE-G2 was tested both as a hub and spoke router. Also, the ISR C3845 platform was added as a spoke router. 3.2 MPLSoL2 Test Strategy NG WAN tests were validated in manual and automated regression testing. The sustaining team takes over regression scripts to continue validation efforts by executing the scripts for any new IOS release and platform. Manual and automated regression test-beds have the same topology, platforms and hardware coverage. Key aspects of the testing methodology: • System validation of advanced MPLS/L3VPN features, such as QoS, mVPN • Interoperability among multiple Cisco platforms, interfaces, and IOS releases • Validation of successful deployment of real applications (Cisco IP Telephony and IPTV multicast video streams) in the network. • End-to-End system validation of all the solutions together in a single integrated customer representative network 3.2.1 MPLSoL2 Test Topology The MPLSoL2 testbed has three major blocks: Campus, MAN and Branch sites. The Cisco Unified Communications Manager and the IPTV server were connected in the campus site and Cisco IP Phones, IPTV Viewer, and PC clients were connected in both campus and branch sites for verifying services. The design of the MAN network was built based on recommendations from the Next Generation Enterprise MPLS VPN-Based MAN Design and Implementation Guide. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 3-3 Chapter 3 WAN Edge: MPLSoL2 Service 3.2.2 Test Types Figure 3-2 MPLSoL2 Test Bed Campus Site M IPTV IP CCM PE1 PE2 RR1 RR2 MAN P P MPL SoL2 SP L2 Service OC-48 OC-12 OC-3 10 GE GE T1 Traffic Generator Branch Sites 3.2.2 Test Types Validation tests were divided into the following types: • System Integration • Scalability • Negative Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 3-4 185836 IP Chapter 3 WAN Edge: MPLSoL2 Service 3.2.3 Sustaining Coverage • Reliability For general descriptions of these test types refer to Appendix D. The following sections describe the specific areas that are covered in each test type for the MPLSoL2 deployment model. 3.2.2.1 System Integration Test The System Integration Test combined all the features required for the MPLSoL2 deployment model. End to End services validation was performed for Branch to Branch and Campus to Branch traffic flows. The services validated include: Multicast using IPTV viewer, IP Telephony using Cisco IP Phones and data connectivity. 3.2.2.2 Scalability Test The MPLSoL2 solution must support at least 500 remote peers in any single hub. Background traffic should be enough to congest the T1 links on the branch router and OC3 link on Hub router. During the scalability test, all the test cases in the System Integration Test suite must continue to function for the duration of the test. A traffic generator was used to simulate 500 spokes with LDP/OSPF/MP-iBGP sessions enabled in each spoke. Data traffic of various rates and DSCP settings was generated on all 500 spokes using the traffic generator. 3.2.2.3 Negative Test All negative test cases were grouped together for better test management. During each of the negative tests, traffic was fully-loaded and the CPU and memory usage of the test-bed was monitored. The negative tests were categorized into the following failure scenarios: • Redundancy/HA: primary hub router/link failover.(Reload/shut/no shut the primary router and links) • Hardware: – LC/SIP/SPA/PA and cable OIR (Online Insertion and Remover) – Router reload • Control-plane: Clear routing tables, clear LDP neighbors etc. 3.2.2.4 Reliability Test A 150-hr reliability test case is performed after all the system integration, negative and scalability testing passed. The system integration test cases are running during this reliability testing. A certain set of parameters such as CPU and memory usage are monitored during the testing to make sure there are no long term memory leaks or excessive CPU consumption. 3.2.3 Sustaining Coverage All the test cases in the System Integration Test suite are included in the automation scripts. The Enterprise NG-WAN System test team developed the automation test solutions which include following components: Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 3-5 Chapter 3 WAN Edge: MPLSoL2 Service 3.3 MPLSoL2 Hardware and Software Information • The automated test scripts for each automation test cases • The common library for managing the test-bed, collecting and reporting the test results • The automated procedures to capture the manual execution results All the real applications used in the manual validation phase, including IPTV server/client, Cisco Unified Communications Manager server and IP phones, were not automated. Instead, traffic tools were used to generate simulated traffic such as voice and video on the network. 3.3 MPLSoL2 Hardware and Software Information Table 3-1 MPLSoL2 Hardware and Software Information Hardware Platform Role Software Version LIne Cards/Interfaces Customer's Edge router 12.2(18)SXF7 SUP720-3BXL, WS-6724-SFP Campus Cisco 6500 (CE) (CE1 and CE2) MAN Cisco 6500 (P) Provider's core router (P1) 12.2(18)SXF7 SUP720-3BXL, SIP-600, SPA-GE, WS-6724-SFP Cisco 7600 (P) Provider's core router (P2) 12.2(33)SRA2 SUP720-3BXL, SIP-600, SPA-GE, SPA-10GE, 6704-10GE, OSM-OC48 Cisco 7600 (PE) Provider's Edge 12.2(33)SRA2 router SUP720-3BXL, SIP-600, SIP-400, SPA-GE, SPA-OC3, SPA-OC12 (PE1) Cisco 7200 (RR) Core router reflector 12.2(31)SB2 NPE-G2/GE 12.4(11)T1 NPE-G2, PA-OC3 12.2(33)SRA2 SUP720-3BXL, SIP-600, SIP-400, SPA-GE, SPA-OC3 12.4(11)T1 NPE-G2, POS-OC3 (RR1 and RR2) WAN Hubs Cisco 7200 (P) WAN hub router (HUB1) Cisco 7600 (P) WAN hub router (HUB2) Branch Routers Cisco 7200 (PE) Branch router (E-PE1) Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 3-6 Chapter 3 WAN Edge: MPLSoL2 Service 3.4 MPLSoL2 Test Results and Recommendations Table 3-1 MPLSoL2 Hardware and Software Information Hardware Platform Role Software Version LIne Cards/Interfaces Cisco C2851 Branch router 12.4(11)T1 T1 12.4(11)T1 T1 (E-PE2) Cisco C3845 Branch router (E-PE3) 3.4 MPLSoL2 Test Results and Recommendations This section presents test results and recommendations for the MPLSoL2 deployment model. 3.4.1 MPLSoL2 Test Results A summary of the test results for the MPLSoL2 deployment model is presented in Table 3-2. For more details on the MPLSoL2 test cases, please refer to B.1 MPLSoL2 Deployment Model. Table 3-2 MPLSoL2 Test Results Summary Test Results Test Types Number of Test Cases Pass Pass with Exception Fail System Integration 8 6 0 2 Negative 3 2 1 0 Scalability 2 2 0 0 Reliability 1 1 0 0 Total 14 1 1 2 The two system integration failures were due to software defect CSCsj78913. This failure affects Multicast VPN traffic on a Cisco 2851 ISR router running Cisco IOS version 12.4(11)T. CSCsj78913 was not observed on later IOS version, 12.4(15)T1, and is now marked as "not reproducible". For more details about this defect see CSCsj78913. One of the negative test cases was considered pass with exception due to software defect CSCsi44003. This particular negative test simulates a spoke router failure by reloading one of the C3845 Branch routers. During this test, the CLI (command line interface) "mtu 1508" under a subinterface disappears after the router reloads. As a result, OSPF neighbor adjacency is not re-established due to MTU mismatch. This defect is only observed in spoke routers with a particular interface card (VWIC-2MFT-T1-DI). For more details on the defect please see CSCsi44003. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 3-7 Chapter 3 WAN Edge: MPLSoL2 Service 3.4.2 MPLSoL2 Recommendations 3.4.2 MPLSoL2 Recommendations Further regression testing was done on Cisco IOS release 12.4(15)T1 and the MPLSoL2 test suite passed. Because of a pending Field Notice on 12.4T release that may result in router reload and performance degradation, it is recommended to check for regression results for Cisco IOS release 12.4(15)T2. This field notice only affects the hub (C7200) and spoke routers listed in Section 3.3 The MPLSoL2 deployment model is considered Passed with Exception until an image with the resolution of the QoS Field Notice and completely passes the MPLSoL2 regression test suite. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 3-8 CH A P T E R 4 WAN Edge: DMVPN per VRF Revised: October 23, 2007 The DMVPNperVRF deployment model is another option for branch virtualization described in the Next Generation Enterprise MPLS-Based WAN Design and Implementation Guide (CVD I). A brief overview of the solution, test coverage, test strategy and a summary of the results with recommendations are presented in this section. 4.1 DMVPN per VRF Test Coverage The DMVPN per VRF deployment model can be used over a Layer 2 or Layer 3 service from a provider. If it is a Layer 3 VPN service, then the enterprise purchases only a single VPN from the provider but overlays its own VPN's by using a combination of Multi-VRF and GRE. The headend has an mGRE tunnel per VRF, the branches have either GRE (if no spoke-to-spoke communications is required) or mGRE (if spoke-to-spoke communication is required) tunnel per VRF. By configuring, mGRE on certain spokes, it provides them the ability to create dynamic tunnels to other spokes (which should be configured with mGRE as well) on a per-VRF basis. Most enterprises only have a partial mesh requirement - large sites need to be meshed together but the smaller sites are typically only hub and spoke. Thus the deployment is expected to be a combination of GRE and mGRE at the spokes - see Figure 4-1 on page 4-2. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 4-1 Chapter 4 WAN Edge: DMVPN per VRF 4.1.1 DMVPN per VRF Feature Coverage Figure 4-1 DMVPN per VRF RR MPLS MAN MP-iBGP for VPNv4 routes VRF per GRE per VRF SP Network Multi-VRF CE Remote Multi-VRF CE Multi-VRF CE IGP per VRF 185838 Multi-VRF CE 4.1.1 DMVPN per VRF Feature Coverage The feature set in this deployment model closely follows what is recommended in the CVD I document. The testbed configuration is using a redundant hub scenario. The following key features were tested; • Dynamic Multipoint VPN (DMVPN) • Next Hop Resolution Protocol (NHRP) • Multi-VRF Support (VRF lite) • OSPF PE-CE routing protocol • MPBGP (Multiprotocol BGP) • BGP • MPLS Label Distribution Protocol (LDP) • MPLS Virtual Private Network • OSPF Support for Multi-VRF • Multicast VPN (mVPN) • QoS: LLQ, CBWFQ, MPLS QoS, WRED • Source Specific Multicast (SSM) • Multicast over NBMA Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 4-2 Chapter 4 WAN Edge: DMVPN per VRF 4.1.2 CVD II Additional Coverage • Redundancy 4.1.2 CVD II Additional Coverage The C7200 with the NPE-G2 was tested both as a hub and spoke router. Also, the ISR C3845 platform was added as a spoke router and the C6500 with SUP-720-3BXL as a second hub router. 4.2 DMVPN per VRF Test Strategy NG WAN tests were validated in manual and automated regression testing. The sustaining team takes over regression scripts to continue validation efforts by executing the scripts for any new IOS release and platform. Manual and automated regression test-beds have the same topology, platforms and hardware coverage. Key aspects of the testing methodology: • System validation of advanced MPLS/L3VPN features, such as QoS, mVPN • Interoperability among multiple Cisco platforms, interfaces, and IOS releases • Validation of successful deployment of real applications (Cisco IP Telephony and IPTV Multicast video streams) in the network. • End-to-End system validation of all the solutions together in a single integrated customer representative network 4.2.1 DMVPN per VRF Test Topology The DMVPN per VRF testbed has 3 major blocks: Campus, MAN and Branch sites. The Cisco Unified Communications Manager and the IPTV server are connected in the campus site and Cisco IP Phones, IPTV Viewer, and PC clients are connected in both campus and branch sites for verifying services. The design of the MAN network was built based on recommendations from the Next Generation Enterprise MPLS VPN-Based MAN Design and Implementation Guide. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 4-3 Chapter 4 WAN Edge: DMVPN per VRF 4.2.2 Test Types Figure 4-2 DMVPN per VRF Test Bed Campus Site M IPTV IP CCM CE1 CE2 PE1 PE2 RH1 RH2 MAN P P DMVPNper VRF Service Provider OC-48 OC-12 OC-3 10 GE GE T1 500 Simulated Spokes Branch Sites 4.2.2 Test Types Validation tests are divided into the following types: • System Integration • Scalability • Negative Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 4-4 185837 IP Chapter 4 WAN Edge: DMVPN per VRF 4.2.2 Test Types • Reliability For general descriptions of these test types refer to Appendix D. The following sections describe the specific areas that are covered in each test type for the DMVPN per VRF deployment model. 4.2.2.1 System Integration Test The System Integration Test combined all the features required for the DMVPN per VRF deployment model. End to End services validation was performed for Branch to Branch and Campus to Branch traffic flows. The services validated include: Multicast using IPTV viewer, IP Telephony using Cisco IP Phones and data connectivity. 4.2.2.2 Scalability Test The DMVPN per VRF solution must support at least 500 remote peers in any single hub. Background traffic should be enough to congest the T1 links on the branch router and OC3 link on Hub router. During the scalability test, all the test cases in the System Integration Test suite must continue to function for the duration of the test. For DMVPN per VRF there was no third-party tool available to simulate the remote branches. An in-house scalability test-bed was designed to execute the scalability test. The 500 remote branches were simulated by five Cisco C7206VXR with NPE-G2s/VSAs as shown in Figure 4-2 on page 4-4. A traffic generator was connected to the scalability test-bed to generate the bi-directional traffic via the 802.1q trunking interfaces. The following numbers of sessions were simulated on the WAN aggregation hub router during testing: • 500 OSPF neighbors • 500 LDP neighbors • 500 NHRP entries • 500 IKE/IPSec sessions 4.2.2.3 Negative Test All negative test cases were grouped together for better test management. During each of the negative tests, traffic was fully-loaded and the CPU and memory usage of the test-bed was monitored. The negative tests were categorized into the following failure scenarios: • Redundancy/HA: primary hub router/link failover.(Reload/shut/no shut the primary router and links) • Hardware: – LC/SIP/SPA/PA and cable OIR (Online Insertion and Remover) – Router reload • Control-plane: Clear routing tables, clear LDP neighbors etc. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 4-5 Chapter 4 WAN Edge: DMVPN per VRF 4.2.3 Sustaining Coverage 4.2.2.4 Reliability Test A 150-hr reliability test case is performed after all the system integration, negative and scalability testing passed. The system integration test cases are running during this reliability testing. A certain set of parameters such as CPU and memory usage are monitored during the testing to make sure there are no long term memory leaks or excessive CPU consumption. 4.2.3 Sustaining Coverage All the test cases in the System Integration Test suite are included in the automation scripts. The Enterprise NG-WAN System test team developed the automation test solutions which include following components: – The automated test scripts for each automation test cases – The common library for managing the test-bed, collecting and reporting the test results – The automated procedures to capture the manual execution results All the real applications used in the manual validation phase, including IPTV server/client, Cisco Unified Communications Manager server and IP phones, were not automated. Instead, traffic tools were used to generate simulated traffic such as voice and video on the network. 4.3 DMVPN per VRF Hardware and Software Information Table 4-1 DMVPN per VRF Hardware and Software Information Hardware Platform Role Software Version Line Cards/Interfaces Customer's Edge router 12.2(18)SXF7 WS-6724-SFP Campus Cisco 6500 (CE) (CE1 and CE2) MAN Cisco 6500 (P) Provider's core router (P1) 12.2(18)SXF7 SIP-600, SPA-GE, WS-6724-SFP Cisco 7600 (P) Provider's core router (P2) 12.2(33)SRA2 SIP-600, SPA-GE, SPA-10GE, 6704-10GE, OSM-OC48 Cisco 7600 (PE) Provider's Edge router 12.2(33)SRA2 SIP-600, SIP-400, SPA-GE, SPA-OC3, SPA-OC12 12.2(31)SB2 NPE-G2/GE (PE1) Cisco 7200 (RR) Core router reflector (RR1 and RR2) Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 4-6 Chapter 4 WAN Edge: DMVPN per VRF 4.4 DMVPN per VRF Test Results and Recommendations Table 4-1 DMVPN per VRF Hardware and Software Information Hardware Platform Role Software Version Line Cards/Interfaces Cisco 7600 (PE) Provider's Edge router 12.2(33)SRA2 SIP-600, SIP-400, SPA-GE 12.4(11)T1 PA-OC3, NPE-G2/GE 12.2(33)SRA2 SIP-600, SIP-400, SPA-GE, SPA-OC3 12.4(11)T1 POS-OC3, NPE-G2/GE 12.4(11)T1 T1 12.4(11)T1 T1 (PE3) WAN HUBS Cisco 7200 (PE) WAN hub routers acting as PE (HUB1) Cisco 6500 (PE) WAN hub routers acting as PE (HUB2) Branch Routers Cisco 7200 (PE) Branch router (E-PE1) Cisco C2851 Branch router (E-PE2) Cisco C3845 Branch router (E-PE3) 4.4 DMVPN per VRF Test Results and Recommendations This section presents test results and recommendations for the DMVPN per VRF deployment model. 4.4.1 DMVPN per VRF Test Results A summary of the test results for the DMVPN per VRF deployment model is presented in Table 4-2. For more details on the DMVPN per VRF test cases, please refer to B.2 DMVPNperVRF Deployment Model. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 4-7 Chapter 4 WAN Edge: DMVPN per VRF 4.4.2 DMVPN per VRF Recommendations Table 4-2 TDMVPN per VRF Test Results Summary Test Results Test Types Number of Test Cases Pass Pass with Exception Fail System Integration 8 7 1 0 Negative 3 3 0 0 Scalability 1 0 0 1 Reliability 1 1 0 0 Total 13 1 1 1 There is one test case that passed with exception because Multicast over DMVPN is not supported on C6500 platform. The same test case passed on the Cisco C7200 hub router. The second failure found in scalability testing is considered operationally impacting, CSCek74416. During large scale DMVPN per VRF testing, one of the DMVPN spoke experienced a software crash. This defect is resolved but not in a released IOS images. 4.4.2 DMVPN per VRF Recommendations Because of the pending verfication of the fix for CSCek74416, it is recommended to look for DMVPN per VRF regression results starting at Cisco IOS release 12.4(15)T2. This recommendations only applies to the hub router and the spoke routers listed in Section 4.3. For the C6500 with Sup-720-3BXL, please note that there is no Multicast over DMVPN support for C6500 platform. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 4-8 CH A P T E R 5 WAN Edge: MPLS over DMVPN - 2547oDMVPN (Hub and Spoke Only) Revised: October 23, 2007 There are two possible modes in this deployment model that were outlined in the Next Generation Enterprise MPLS-Based WAN Design and Implementation Guide (CVD I); • 2547oDMVPN Hub as a P router • 2547oDMVPN Hub as a PE router The 2547oDMVPN (Hub as a P router) was identified as the preferred mode in CVD I over the 2547oDMVPN (Hub as PE router) but was not validated in CVD I because of the lack of LDP support for mGRE. Support for the 2547oDMVPN (hub as P router) deployment model is now available in the Cisco IOS release 12.4(11) T and test coverage was added as part of the CVD II testing. Validation was focused more on the Hub as a P router mode rather than the 2547oDMVPN (Hub as a PE router). 5.1 2547oDMVPN Test Coverage This model does not have some of the scale limitations of the Multi-VRF based solutions because the GRE tunnels are created outside the VRF's and hence a single tunnel can be shared for transporting many VRF's. The hub is configured with a single mGRE tunnel while the spokes have a single GRE tunnel. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 5-1 Chapter 5 WAN Edge: MPLS over DMVPN - 2547oDMVPN (Hub and Spoke Only) 5.1.1 2547oDMVPN Feature Coverage Figure 5-1 2547oDMVPN (Hub as a P router) RR MPLS MAN IGP and LDP E-P mGRE MP-iBFP for VPNv4 routes GRE E-PE E-PE E-PE E-PE Remote IGP and LDP over GRE 185839 SP Network 5.1.1 2547oDMVPN Feature Coverage The feature set in this deployment model closely follows what is recommended in the CVD I document. The testbed configuration assumes a redundant hub scenario. The following key features were tested; • Dynamic Multipoint VPN (DMVPN) • Next Hop Resolution Protocol (NHRP) • IPSec/IKE • 2547oDMVPN • MPBGP (Multiprotocol BGP) • BGP • BGP Route Reflector • MPLS Label Distribution Protocol (LDP) • MPLS Virtual Private Network • OSPF Support for Multi-VRF • Multicast VPN (mVPN) • QoS: LLQ, CBWFQ, MPLS QoS, WRED • Source Specific Multicast (SSM) Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 5-2 Chapter 5 WAN Edge: MPLS over DMVPN - 2547oDMVPN (Hub and Spoke Only) 5.1.2 CVD II Additional Coverage • Multicast over NBMA • Redundancy • HSRP • OSPF Routing 5.1.2 CVD II Additional Coverage Since the 2547oDMVPN Hub as a P router is available, it was added in CVD II validation efforts. The C7200 with the NPE-G2 was tested both as a Hub and spoke router. Also, the ISR C3845 platform was added as a spoke router. 5.2 2547oDMVPN Test Strategy NG WAN tests were validated in manual and automated regression testing. The sustaining team takes over regression scripts to continue validation efforts by executing the scripts for any new IOS release and platform. Manual and automated regression test-beds have the same topology, platforms and hardware coverage. Key aspects of the testing methodology: • System validation of advanced MPLS/L3VPN features, such as QoS, mVPN • Interoperability among multiple Cisco platforms, interfaces, and IOS releases • Validation of successful deployment of real applications (Cisco IP Telephony and IPTV Multicast video streams) in the network. • End-to-End system validation of all the solutions together in a single integrated customer representative network 5.2.1 2547oDMVPN Test Topology The 2547oDMVPN testbed has 3 major blocks: Campus, MAN and Branch sites. The Cisco Unified Communications Manager and the IPTV server are connected in the campus site and Cisco IP Phones, IPTV Viewer, and PC clients are connected in both campus and branch sites for verifying services. The design of the MAN network was built based on recommendations from the Next Generation Enterprise MPLS VPN-Based MAN Design and Implementation Guide. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 5-3 Chapter 5 WAN Edge: MPLS over DMVPN - 2547oDMVPN (Hub and Spoke Only) 5.2.2 Test Types Figure 5-2 2547oDMVPN Testbed Campus Site M IPTV IP CCM CE1 CE2 PE1 PE2 P MAN P RR RR P P HUB1 HUB2 2547oDMVPN (P) Service Provider E-PE1 E-PE2 E-PE3 OC-48 OC-12 OC-3 10 GE GE T1 500 Simulated Spokes Branch Sites 5.2.2 Test Types Validation tests are divided into the following types: • System Integration • Scalability • Negative Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 5-4 240978 IP Chapter 5 WAN Edge: MPLS over DMVPN - 2547oDMVPN (Hub and Spoke Only) 5.2.2 Test Types • Reliability For general descriptions of these test types refer to Appendix D. The following sections describe the specific areas that are covered in each test type for the 2547oDMVPN deployment model. 5.2.2.1 System Integration Test The System Integration Test combined all the features required for the 2547oDMVPN deployment model. End to End services validation was performed for Branch to Branch and Campus to Branch traffic flows. The services validated include: Multicast using IPTV viewer, IP Telephony using Cisco IP Phones and data connectivity. 5.2.2.2 Scalability Test Scalability testing measures the limit of a particular variable when all others are constant in a system level environment. For example, the number of routing entries that the system can support or number of OSPF neighbors. The 2547oDMVPN solution must support at least 500 remote peers in any single hub. Background traffic should be enough to congest the T1 links on the branch router and OC3 link on Hub router. During the scalability test, all the test cases in the System Integration Test suite must continue to function for the duration of the test. For the 2547oDMVPN deployment model, there was no third-party tool available to simulate the remote branches. An in-house scalability test-bed was designed to execute the scalability test. The 500 remote branches were simulated by five Cisco C7206VXR with NPE-G2s/VSAs as shown in Figure 5-2 on page 5-4. A traffic generator was connected to the scalability test-bed to generate the bi-directional traffic via the 802.1q trunking interfaces. The following numbers of sessions were simulated on the WAN aggregation hub during testing: • 500 OSPF neighbors • 500 LDP neighbors • 500 NHRP entries • 500 IKE/IPSec sessions • 500 MP-iBGP sessions with RR (Router Reflector) 5.2.2.3 Negative Test All negative test cases were grouped together for better test management. During each of the negative tests, traffic was fully-loaded and the CPU and memory usage of the test-bed was monitored. The negative tests were categorized into the following failure scenarios: • Redundancy/HA: primary hub router/link failover.(Reload/shut/no shut the primary router and links) • Hardware: – LC/SIP/SPA/PA and cable OIR (Online Insertion and Remover) – Router reload • Control-plane: Clear routing tables, clear LDP neighbors etc. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 5-5 Chapter 5 WAN Edge: MPLS over DMVPN - 2547oDMVPN (Hub and Spoke Only) 5.2.3 Sustaining Coverage 5.2.2.4 Reliability Test A 150-hr reliability test case is performed after all the system integration, negative and scalability testing passed. The system integration test cases are running during this reliability testing. A certain set of parameters such as CPU and memory usage are monitored during the testing to make sure there are no long term memory leaks or excessive CPU consumption. 5.2.3 Sustaining Coverage All the test cases in the System Integration Test suite are included in the automation scripts. The Enterprise NG-WAN System test team developed the automation test solutions which include following components: • The automated test scripts for each automation test cases • The common library for managing the test-bed, collecting and reporting the test results • The automated procedures to capture the manual execution results All the real applications used in the manual validation phase, including IPTV server/client, Cisco Unified Communications Manager server and IP phones, were not automated. Instead, traffic tools were used to generate simulated traffic such as voice and video on the network. 5.3 2547oDMVPN Hardware and Software Information Table 5-1 2547oDMVPN Hardware Platforms and Data Hardware Platform Role Software Version Line Cards/Interfaces Customer's Edge router 12.2(18)SXF7 WS-6724-SFP Campus Cisco 6500 (CE) (CE1 and CE2) MAN 12400 (P) Provider's core router 12.0(32)S2 SIP-600, SIP-601, SPA-POS-OC48, SPA-10GE, SPA-GE, ISE 4GE, ISE POS-4OC12 Cisco 7600 (P) Provider's core router (P2) 12.2(33)SRA2 SIP-600, SPA-GE, SPA-10GE, 6704-10GE, OSM-OC48 Cisco 7600 (PE) Provider's Edge router 12.2(33)SRA2 SIP-600, SIP-400, SPA-GE, SPA-OC3, SPA-OC12 12.0(32)S3 NPE-G2/GE (PE1) Cisco 7200 (RR) Core router reflector (RR1 and RR2) Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 5-6 Chapter 5 WAN Edge: MPLS over DMVPN - 2547oDMVPN (Hub and Spoke Only) 5.4 2547oDMVPN Test Results and Recommendations Table 5-1 2547oDMVPN Hardware Platforms and Data Hardware Platform Role Software Version Line Cards/Interfaces WAN hub routers 12.4(11)T1 PA-OC3, NPE-G2/GE 12.4(11)T1 POS-OC3, NPE-G2/GE 12.4(11)T1 T1 12.4(11)T1 T1 WAN HUBS Cisco 7200 (P) (HUB1 and HUB2) *see note Branch Routers Cisco 7200 (PE) Branch router (E-PE1) Cisco C2851 Branch router (E-PE2) Cisco C3845 Branch router (E-PE3) Hardware PlatformRoleSoftware VersionLine Cards/Interfaces Note The 2547oDMVPN is currently only supported on the Cisco C7200 platform. 5.4 2547oDMVPN Test Results and Recommendations This section presents test results and recommendations for the 2547oDMVPN deployment model. 5.4.1 2547oDMVPN Test Results A summary of the test results for the 2547oDMVPN (Hub as a P router) deployment model is presented in Table 5-2. For more details on the 2547oDMVPN test cases, please refer to B.4 2547oDMVPN (Hub as P Role) Deployment Model. Table 5-2 25470DMVPN Test Results Summary Test Results Test Types Number of Test Cases Pass Pass with Exception Fail System Integration 7 7 0 0 Negative 2 1 0 1 Scalability 1 1 0 0 Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 5-7 Chapter 5 WAN Edge: MPLS over DMVPN - 2547oDMVPN (Hub and Spoke Only) 5.4.2 2547oDMVPN Recommendations Table 5-2 25470DMVPN Test Results Summary Test Results Test Types Number of Test Cases Pass Pass with Exception Fail Reliability 1 1 0 0 Total 11 10 0 1 All the system integration, scalability and reliability test cases in this model passed. There was one failure in the negative testing. The failure was observed when simulating a redundant hub router reload. This particular defect fails to restore mVPN (Multicast VPN) feature and consequently Multicast streams between campus and branch fail. For more details, see CSCsi49487. This defect turned out to be not a problem with the Hub router but with the specific IOS release used for the Route Reflector(RR) , 12.0(32)S3. Once the route reflector IOS release is upgraded to 12.2(31)SB2, the test case passed. 5.4.2 2547oDMVPN Recommendations Further regression testing was done on Cisco IOS release 12.4(15)T1 and the 2547oDMVPN test suites passed. Because of a pending Field Notice on 12.4T release that may result in router reload and performance degradation, it is recommended to check for regression results for Cisco IOS release 12.4(15)T2. This field notice only affects the hub (C7200) and spoke routers listed in Section 5.3 The 2547oDMVPN deployment model is considered Passed with Exception until an image with the resolution of the QoS Field Notice and completely passes the 2547oDMVPN regression test suite. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 5-8 CH A P T E R 6 WAN Core: Inter-AS, ASBR-to-ASBR with MPeBGP Revised: October 23, 2007 In the Next Generation Enterprise MPLS-Based WAN Design and Implementation Guide (CVD I), there are a number of possible solutions mentioned for interconnecting MPLS MAN networks: • MPLSoL2 service • Inter-AS • MPLSoGRE • Carrier Supporting Carrier (CSC). The Inter-AS (ASBR-to-ASBR with MPeBGP) was selected to be the method of interconnecting the two MPLS networks because it was considered more scalable and widely deployed. There are no specific implementation recommendations in the design guide with regards to configuring and deploying inter-AS, so the CVD II validation efforts provided additional coverage for this deployment model. 6.1 Inter-AS Test Coverage If the MAN islands and campuses are under different administrative control, Inter-AS can be implemented. Apart from being a simple solution to deploy, it also offers the wider platform options. All the platforms that support P roles should be deployable. All the features that would be deployed within a MPLS network (such as TE) can also be deployed across the WAN core. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 6-1 Chapter 6 WAN Core: Inter-AS, ASBR-to-ASBR with MPeBGP 6.1.1 Inter-AS Feature Coverage Figure 6-1 Inter-AS (ASBR-to-ASBR with MPeBGP) Inter-AS MAN1 M1=ASBR MAN2 M2=ASBR E-PE E-PE RR RR SP E-P M1=ASBR E-P M2=ASBR E-PE 185840 E-PE E-PE 6.1.1 Inter-AS Feature Coverage The following key features were tested: • Inter-AS Multicast • Inter-AS Unicast • Redundancy • OSPF • BGP • MPLS Multiprotocol Label Switching • Inter-AS BGP - MPLS VPN • BGP interAS MVPN support • MPLS VPN Load balancing support for InterAS & CSC • InterAS support for MVPN • MPLS VPN - Multi-Path support for Inter-AS VPNs • MPLS VPN Inter-AS - IPv4 BGP Label Distribution 6.1.2 CVD II Additional Coverage This deployment model was not covered in the CVD I and was added as part of CVD II validation. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 6-2 Chapter 6 WAN Core: Inter-AS, ASBR-to-ASBR with MPeBGP 6.2 Inter-AS Test Strategy 6.2 Inter-AS Test Strategy NG WAN tests were validated in manual and automated regression testing. The sustaining team takes over regression scripts to continue validation efforts by executing the scripts for any new IOS release and platform. Manual and automated regression test-beds have the same topology, platforms and hardware coverage. Key aspects of the testing methodology: • System validation of advanced MPLS/L3VPN features, such as QoS, mVPN • Interoperability among multiple Cisco platforms, interfaces, and IOS releases • Validation of successful deployment of real applications (Cisco IP Telephony and IPTV Multicast video streams) in the network. • End-to-End system validation of all the solutions together in a single integrated customer representative network 6.2.1 Inter-AS Test Topology The Inter-AS testbed has two MAN and one campus network attached to each MAN. The Cisco Unified Communications Manager and the IPTV server are connected in the campus sites. Cisco IP Phones, IPTV Viewer, and PC clients are connected in both campus sites for verifying services. The design of the MAN network was built based on recommendations from the Next Generation Enterprise MPLS VPN-Based MAN Design and Implementation Guide. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 6-3 Chapter 6 WAN Core: Inter-AS, ASBR-to-ASBR with MPeBGP 6.2.1 Inter-AS Test Topology Figure 6-2 Inter-AS (ASBR-to-ASBR with MPeBGP) Testbed Campus 1 M MAN1 IP PE PE P P RR RR P ASBR P ASBR Inter AS MAN2 ASBR ASBR P P P P PE PE M Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 6-4 IP 185841 Campus 2 Chapter 6 WAN Core: Inter-AS, ASBR-to-ASBR with MPeBGP 6.2.2 Test Types 6.2.2 Test Types Validation tests are divided into the following types: • System Integration • Negative For general descriptions of these test types refer to Appendix D. The following sections describe the specific areas that are covered in each test type for the Inter-AS deployment model. 6.2.2.1 System Integration Test The System Integration Test combined all the features required for the Inter-AS deployment model. End to End services validation was performed for Campus to Campus traffic flows. The services validated include: Multicast using IPTV viewer, IP Telephony using Cisco IP Phones and data connectivity. 6.2.2.2 Negative Test All negative test cases were grouped together for better test management. During each of the negative tests, traffic was fully-loaded and the CPU and memory usage of the test-bed was monitored. The negative tests were categorized into the following failure scenarios: • Redundancy/HA: primary ASBR router/link failover.(Reload/shut/no shut the primary router and links) • Hardware: Router reload • Control-plane: clear routing tables 6.2.3 Sustaining Coverage All the test cases in the System Integration Test suite are included in the automation scripts. The Enterprise NG-WAN System test team developed the automation test solutions which include following components: • The automated test scripts for each automation test cases • The common library for managing the test-bed, collecting and reporting the test results • The automated procedures to capture the manual execution results All the real applications used in the manual validation phase, including IPTV server/client, Cisco Unified Communications Manager server and IP phones, were not automated. Instead, traffic tools were used to generate simulated traffic such as voice and video on the network. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 6-5 Chapter 6 WAN Core: Inter-AS, ASBR-to-ASBR with MPeBGP 6.3 Inter-AS Hardware and Software Information 6.3 Inter-AS Hardware and Software Information Table 6-1 Inter-AS Hardware and Software Information Hardware Platform Role Software Version Line Cards/Interfaces Customer's Edge router (CE1 and CE2) 12.2(18)SXF7 WS-6724-SFP Autonomous System Border Router 12.0(32)S2, 12.0(32)SY SIP-601, SPA-POS-OC48, SPA-5XGE Cisco 7600 (ASBR) Autonomous System Border Router 12.2(33)SRA2 SIP-600, SIP-400, SPA-5XGE, SPA-POS-OC48 Cisco 6500 (P) Provider's core router (P1) 12.2(18)SXF7 SIP-600, SPA-GE, WS-6724-SFP Cisco 7600 (P) Provider's core router (P2) 12.2(33)SRA2 SIP-600, SPA-GE, SPA-10GE, 6704-10GE, OSM-OC48 Cisco 7600 (PE) Provider's Edge router (PE1) 12.2(33)SRA2 SIP-600, SIP-400, SPA-GE, SPA-OC3, SPA-OC12 Cisco 7200 (RR) Core router reflector (RR1 and 12.2(31)SB2 RR2) NPE-G2/GE Autonomous System Border Router 12.0(32)S2, 12.0(32)SY SIP-601, SPA-POS-OC48, SPA-5XGE Cisco 7600 (ASBR) Autonomous System Border Router 12.2(33)SRA2 SIP-600, SIP-400, SPA-5XGE, SPA-POS-OC48 Cisco 6500 (P) Provider's core router (P1) 12.2(18)SXF7 SIP-600, SPA-GE, WS-6724-SFP Cisco 7600 (P) Provider's core router (P2) 12.2(33)SRA2 SIP-600, SPA-GE, SPA-10GE, 6704-10GE, OSM-OC48 Cisco 7600 (PE) Provider's Edge router (PE1) 12.2(33)SRA2 SIP-600, SIP-400, SPA-GE, SPA-OC3, SPA-OC12 Cisco 7200 (RR) Core router reflector (RR1 and 12.2(31)SB2 RR2) Campus Cisco 6500 (CE) MAN1 12400 (ASBR) MAN2 12400 (ASBR) Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 6-6 NPE-G2/GE Chapter 6 WAN Core: Inter-AS, ASBR-to-ASBR with MPeBGP 6.4 Inter-AS Test Results and Recommendations 6.4 Inter-AS Test Results and Recommendations This section presents test results and recommendations for the Inter-AS deployment model. 6.4.1 Inter-AS Test Results The Inter-AS test suite passed all system integration and negative test cases. A summary of the test results for the Inter-AS deployment model is presented in Table 6-2. For more details on the Inter-AS test cases, please refer to B.5 Inter-AS Deployment Model. Table 6-2 Inter-AS Test Results Summary Test Results Test Types Number of Test Cases Pass Pass with Exception Fail System Integration 5 5 0 0 Negative 3 3 0 0 Total 8 8 0 0 6.4.2 Inter-AS Recommendations There were no defects encountered or discovered for the Inter-AS deployment model. Please refer to Table 6-1 for recommended platform and IOS releases. This deployment model is being maintained by a sustaining regression team. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 6-7 Chapter 6 6.4.2 Inter-AS Recommendations Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 6-8 WAN Core: Inter-AS, ASBR-to-ASBR with MPeBGP CH A P T E R 7 References Revised: October 23, 2007 Next Generation Enterprise MPLS-Based WAN Design and Implementation Guide (EDCS-536219) http://www.cisco.com/application/pdf/en/us/guest/products/ps6892/c1244/ccmigration_09186a00808c e6ad.pdf Next Generation Enterprise MPLS VPN-Based MAN Design and Implementation Guide(EDCS-468156) http://www.cisco.com/application/pdf/en/us/guest/netsol/ns241/c649/ccmigration_09186a008055edcf. pdf NSITE Next Generation Enterprise MPLS-based WAN Test Results Report (EDCS-625000) Available upon request from your Cisco System Engineer.) Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 7-1 Chapter 7 Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II 7-2 References A P P E N D I X A Test Coverage Matrix Revised: October 23, 2007 A.1 NG WAN Test Coverages Matrix Table A-1 below compares what was covered in CVDI versus CVDII. The table lists all the features, platforms, and software versions for all the deployment models. Table A-1 MPLSoL2 Features MPLSoL2 Deployment Architecture CVDI CVDII Features X X MPLS VPN X X MPLS Label Distribution Protocol (LDP) X X MPBGP (Multiprotocol BGP) X X Multicast VPN (mVPN) X X QoS: LLQ, CBWFQ, MPLS QoS, WRED X X Source Specific Multicast (SSM) X X Multicast over NBMA X X Redundancy X X Table A-2 MPLSoL2 CVDI Platforms and Software Role Platform Software Hub C7200 12.4(6)T Hub C12000 12. Hub C7600 12.2(18)SXF Spoke ISR's 12.4(6)T Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II A-1 Appendix A Test Coverage Matrix A.1 NG WAN Test Coverages Matrix Table A-3 MPLSoL2 CVDII Platforms and Software Role Platform Software Hub C7200-G2 12.4(11)T Hub C7600 12.2(33)SRA2 Spoke C7200-G2 12.4(11)T Spoke C2851 12.4(11)T Spoke C3845 12.4(11)T Table A-4 DMVPNperVRF Features DMVPNperVRF Deployment Architecture CVDI CVDII Features X X Dynamic Multipoint VPN (DMVPN) X X Next Hop Resolution Protocol (NHRP) X X Multi-VRF Support (VRF lite) X X OSPF PE-CE routing protocol X X MPBGP (Multiprotocol BGP) X X BGP X X MPLS Label Distribution Protocol (LDP) X X MPLS Virtual Private Network X X OSPF Support for Multi-VRF X X Multicast VPN (mVPN) X X QoS: LLQ, CBWFQ, MPLS QoS, WRED X X Source Specific Multicast (SSM) X X Multicast over NBMA X X Redundancy X X Table A-5 DMVPNperVRF CVDI Platforms and Software Role Platform Software Hub C7200-G1 12.4(6)T Hub C7600 SUP-720-3BXL 12.2(18)SXF Spoke C2851 and C3825 12.4(6)T Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II A-2 Appendix A Test Coverage Matrix A.1 NG WAN Test Coverages Matrix Table A-6 DMVPNperVRF CVDII Platforms and Software Role Platform Software Hub C7200-G2 12.4(11)T Hub C7600 12.2(33)SRA2 Spoke C7200-G2 12.4(11)T Spoke C2851 12.4(11)T Spoke C3845 12.4(11)T Table A-7 2547oDMVPN (Hub as PE Role) Features 2547oDMVPN (Hub as PE Role) Deployment Architecture Features CVDI CVDII Dynamic Multipoint VPN (DMVPN) X X Next Hop Resolution Protocol (NHRP) X X IPSec/IKE X X 2547oDMVPN X X MPBGP (Multiprotocol BGP) X X BGP X X BGP Route Reflector X X MPLS Label Distribution Protocol (LDP) X X MPLS Virtual Private Network X X OSPF Support for Multi-VRF X X Multicast VPN (mVPN) X X QoS: LLQ, CBWFQ, MPLS QoS, WRED X X Source Specific Multicast (SSM) X X Multicast over NBMA X X Redundancy X X HSRP X X EIGRP Routing NO NO OSPF Routing X X Table A-8 2547oDMVPN (Hub as PE Role) CVDI Platforms and Software Role Platform Software Hub C7200-G1 12.4(6)T Spoke C2851 and C3825 12.4(6)T Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II A-3 Appendix A Test Coverage Matrix A.1 NG WAN Test Coverages Matrix Table A-9 2547oDMVPN (Hub as PE Role) CVDII Platforms and Software Role Platform Software Hub C7200-G2 12.4(11)T Spoke C7200-G2 12.4(11)T Spoke C2851 12.4(11)T Spoke C3845 12.4(11)T Table A-10 2547oDMVPN (Hub as P Role) Features 2547oDMVPN (Hub as P Role) Deployment Architecture Features CVDII Dynamic Multipoint VPN (DMVPN) X Next Hop Resolution Protocol (NHRP) X IPSec/IKE X 2547oDMVPN X MPBGP (Multiprotocol BGP) X BGP X BGP Route Reflector X MPLS Label Distribution Protocol (LDP) X MPLS Virtual Private Network X OSPF Support for Multi-VRF X Multicast VPN (mVPN) X QoS: LLQ, CBWFQ, MPLS QoS, WRED X Source Specific Multicast (SSM) X Multicast over NBMA X Redundancy X HSRP X EIGRP Routing NO OSPF Routing Table A-11 X X 2547oDMVPN (Hub as P Role) CVDII Platforms and Software Role Platform Software Hub C7200-G2 12.4(11)T Spoke C7200-G2 12.4(11)T Spoke C2851 12.4(11)T Spoke C3845 12.4(11)T Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II A-4 CVDI Appendix A Test Coverage Matrix A.1 NG WAN Test Coverages Matrix Table A-12 Inter-AS (MAN CORE connection)Features Inter-AS (MAN CORE connection) Deployment Architecture CVDI CVDII Inter-AS Multicast X Inter-AS Unicast X Redundancy X OSPF X BGP X MPLS Multiprotocol Label Switching X Multi-protocol BGP - MPLS VPN X BGP interAS MVPN support X MPLS VPN Load balancing support for InterAS & CSC X InterAS support for MVPN X MPLS VPN - Multi-Path support for Inter-AS VPNs X MPLS VPN Inter-AS - IPv4 BGP Label Distribution X Table A-13 Inter-AS (MAN CORE Connection) CVDII Platforms and Software Role Platform Software ASBR C12000 12.0(32)SY, 12.0(32)S2 ASBR C7600 12.2(33)SRA2 Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II A-5 Appendix A A.1 NG WAN Test Coverages Matrix Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II A-6 Test Coverage Matrix A P P E N D I X B Test Case Descriptions and Results Revised: October 23, 2007 B.1 MPLSoL2 Deployment Model Table B-1 Test MPLSoL2 Deployment Model Manual Test Case Defects Automation Test Case Defects System Integration Test Suites MPLSoL2: IP Baseline Test Pass — Pass — This test case covers IP infrastructure build up and test under MPLSoL2 solution model. The following will be covered: The 7200 hub router config 7200 Branch router config 3845 branch router config 2851 branch router config QoS config on all test platform Multicast config on all test platform Once MPLSoL2 baseline is up, data/voice/video traffic will be verified among branch and campus. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II B-1 Appendix B Test Case Descriptions and Results B.1 MPLSoL2 Deployment Model MPLSoL2: Branch to Campus Data Communication Pass — Pass — Pass — Pass — — Pass — Pass — This test case covers data communications between branch and campus under MPLSoL2 solution model. Once MPLSoL2 baseline is up, data traffic will be verified between branch and campus. Test will focus on verifying: 1. Manually verify Data traffic between a Branch PC client to a campus server. 2. Verify QoS 3. Using test tool verify that data traffic between branch and campus meets the Service Level Requirements MPLSoL2: Branch to Branch Data Communication This test case covers data communications between branch and branch under MPLSoL2 solution model. After the MPLSoL2 baseline is up, data traffic will be verified between branch and branch. Manually verify data traffic between a branch PC client to a PC server in another branch. MPLSoL2: Branch to Campus Voice Pass Communication This test case covers voice traffic verification between branch and campus. It includes both manual voice verification using Cisco IP phones and simulated IP phones using CallGen for automated testing. Test coverage includes verification of critical voice parameters like one-way latency, average one-way jitter, and packet loss using either IxChariot or IxLoad. The same tool will be used to send Voice call signaling traffic. MPLSoL2: Branch to Branch Voice Communication Pass This test case covers voice traffic verification between branch and branch. It includes both manual voice verification using Cisco IP phones and simulated IP phones using CallGen for automated testing. Test coverage includes verification of critical voice parameters like one-way latency, average one-way jitter, and packet loss using either IxChariot or IxLoad. The same tool will be used to send voice call signaling traffic. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II B-2 Appendix B Test Case Descriptions and Results B.1 MPLSoL2 Deployment Model Pass — Fail CSCsj78913 Using test tool to verify that interactive video and streaming video meets the Service Level requirements. MPLSoL2: Branch-to-campus mixed traffic Pass — Fail CSCsj78913 Pass — MPLSoL2: Campus to Branch Video Communication The purpose of this test case is to verify video traffic in a VRF traversing between branch and campus that are connected to a single Multicast VPN (mVPN) enabled MPLS MAN cloud under MPLSoL2 solution. mVPN control plane verification on PE routers. Branch to campus multicast control plane verification over the mVPN MPLS MAN network. Manually verifying video reception using an IP TV viewer in branch receiving a multicast, unicast stream from IP TV server in campus network across mVPN MPLS MAN network. This test case will set up and verify the MPLSoL2 solution with mixed traffic can successfully be sent from branch-to-campus. The remote branch routers will be configured as PE router so the VPN defined in the large MPLS campus can be delivered to the branch via the MPLSoL2 solution. Both real application (Cisco CCM/IP phone, IPTV server/clients) and simulated voice and video traffic from Ixia will be used. Essentially this test case is a combination of the data, voice, and video test cases to verify them can work together. MPLSoL2: Branch-to-Branch Mixed Traffic Pass This test case will set up and verify the MPLSoL2 solution with mixed traffic can successfully be sent from branch-to-branch. The remote branch routers will be configured as PE router so the VPN defined in the large MPLS campus can be delivered to the branch via the MPLSoL2 solution. Both real application (Cisco CCM/IP phone, IPTV server/clients) and simulated voice and video traffic from Ixia will be used. Essentially this test case is a combination of the data, voice, and video test cases to verify them can work together. Scalability Test Suite Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II B-3 Appendix B Test Case Descriptions and Results B.2 DMVPNperVRF Deployment Model MPLSoL2: LDP Session Scale Between the Hub Pass Router and Branch PE This case tests the scalability on hub router. The hub router scalability decides the overall number of branch MPLSoL2 solution can support. The number of LDP sessions that it can support will affect the system performance and stability. MPLSoL2:BGP Peering Scale from Branch PE Pass This case covers scale test for BGP peering between branch PE and RR. Typically, there are large number of branches (up to the thousands) and with each one peering directly to the core RR. The number of BGP peering between branch PE and core RR will affect the number of branch MPLSoL2 solution can support. Negative Test Suite MPLSoL2: Link Failure Between the Hub and Pass Branch PE This case covers the branch PE router failure negative test. The branch PE router failure will bring down the branch connection to the hub. After coming back, the branch connection should be restored and traffic will be back to normal.. Pass MPLSoL2: The Hub Failure and Recover This case the hub router failure negative test. The hub failure will bring down all branch connection. After coming back, all branch connections should be restored and traffic will be back to normal. Passed with MPLSoL2:the branch PE router Failure and Exception recover — N/A N/A — N/A N/A — N/A N/A N/A N/A — N/A N/A CSCsi44003 N/A N/A This case link failure negative test. The link failures occur between the hub and each branch PE. B.2 DMVPNperVRF Deployment Model Table B-2 Test System Integration Test Suites DMVPNperVRF Deployment Model Manual Test Case Defects Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II B-4 Automation Test Case Defects Appendix B Test Case Descriptions and Results B.2 DMVPNperVRF Deployment Model DMVPN per VRF Baseline Pass — Pass — — Pass — — Pass — This system integration test case is to set up and test the infrastructure for DMVPN per VRF enterprise solution. Control Plane verification. DMVPN Tunnel setup on VRF tunnel interfaces with and without encryption. VRF aware OSPF for IGP over dmvpn tunnel interfaces. Spoke to Spoke dynamic DMVPN tunnel creation. C7200 and C7600 as the WAN hub, and c7200 and c2800/c3800 as the branch routers. DMVPN per VRF: Branch to Campus Data Pass Traffic Verification The purpose of this test case is to verify data traffic in a VRF traversing between DMVPN in a VRF Branch and Campus, which are connected to MPLS MAN cloud. The test will focus on verifying: 1. Manually verify Data traffic between a Branch PC client to a Campus Data center server. 2. Verify QoS 3. Using test tool verify that data traffic meets the Service Level Requirements DMVPN per VRF: Branch to Campus Video Pass with Traffic Verification Exception The purpose of this test case is to verify video traffic C7600 and c6500 in a VRF traversing between Branch and Campus does not support which are connected to a single Multicast VPN Multicast over (mVPN) enabled MPLS MAN cloud. DMVPN The test will focus on verifying: 1. mVPN control plane verification on PE routers 2. Branch to Campus Multicast control plane verification over the mVPN MPLS MAN network. 3. Manually verify Video reception using an IP TV viewer in Branch receiving a multicast, unicast stream from IP TV DMVPN per VRF: Branch-to-Campus Voice Pass Traffic Verification Pass This test case covers voice traffic verification between DMVPN per VRF branch to campus over a MPLS MAN cloud. Manual voice verification using Cisco IP phones. Simulated IP_Phones using CallGen for automated testing. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II B-5 Appendix B Test Case Descriptions and Results B.2 DMVPNperVRF Deployment Model DMVPN per VRF: Branch to Campus converged Pass with exception — IP traffic verification C7600 and c6500 does not support The purpose of this test case is to verify simultaneous data, voice & Video traffic (converged Multicast over IP traffic) in a VRF traversing between DMVPN in DMVPN a VRF Branch and Campus, which are connected to MPLS MAN cloud. Pass — — Pass — — Pass — — Pass — CSCek74416 N/A N/A N/A N/A The test will focus on verifying: 1. Using test tool verify that data, voice, Video traffic meets the Service Level Requirements. 2. While the Traffic tools are sending traffic, Verify the data, voice & Video quality manually. DMVPN per VRF: Branch to Branch Data Pass Traffic Verification The purpose of this test case is to verify data traffic in a VRF traversing on a dynamic tunnel between DMVPN in a VRF Branch to another branch. The test will focus on verifying: 1. Manually verify Data traffic between a Branch PC client to another Branch PC client. 2. Verify QoS Branch to Branch Data path 3. Using test tool verify that data traffic meets the Service Level Requirements. DMVPN per VRF: Branch-to-Branch Voice Pass Traffic Verification This test case covers voice traffic verification between DMVPN per VRF branch to campus over a MPLS MAN cloud. Manual voice verification using Cisco IP Phones. Simulated IP_Phones using CallGen for automated testing. DMVPN per VRF: Branch to Branch converged Pass IP (data & Voice) traffic verification The purpose of this test case is to verify simultaneous data, and voice traffic (converged IP traffic) in a VRF traversing between DMVPN in a VRF branch and campus, which are connected to MPLS MAN cloud. Using test tool to verify that data, voice, video traffic meets the Service Level requirements. Scalability Test Suite DMVPN per VRF: Multiple Branches to Campus Fail Traffic verification. This test case is to Scale the DMVPN per vrf enterprise solution. Negative Test Suite Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II B-6 Appendix B Test Case Descriptions and Results B.3 2547oDMVPN (Hub as PE Role) Deployment Model DMVPN per VRF: Hub Redundancy failover Pass Pass — N/A N/A Pass — N/A N/A Pass — N/A N/A This test case focuses on DMVPN Hub redundancy on the dual hub scenario. Will measure the impact of failover of the primary / active router in different roles. Measure the convergence time during a DMVPN tunnel switchover due to the primary hub failure. DMVPN per VRF: Hub Links Failures This test case is to measure the convergence time during link failure. DMVPN per VRF: Provisioning and un-provisioning This negative test case focuses on to measure the network impact during provisioning and un-provisioning. B.3 2547oDMVPN (Hub as PE Role) Deployment Model Table B-3 Test System Integration Test Suites 2547oDMVPN (Hub in PE Role): Branch-to-Campus Baseline and Data Traffic 2547oDMVPN (Hub as PE Role) Deployment Model Manual Test Case Defects Automation Test Case Defects Pass Pass — — This test case will set up and verify that the 2547oDMVPN (Hub used as a PE router) data traffic can be sent successfully between branch-to-campus. Both the hub and the remote branch routers will be configured as PE routers in a PE-PE topology. The hub PE will also connect to the MPLS core in a PE-PE configure. Thus, the VPN defined in the large MPLS campus can be delivered to the branch via this solution. The DMVPN provided the flexible overlay model, plus the security the WAN solution needed. C7200 will be tested as the WAN hub, and c7200 and c2800/c3800 will be tested as the branch routers. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II B-7 Appendix B Test Case Descriptions and Results B.3 2547oDMVPN (Hub as PE Role) Deployment Model 2547oDMVPN(PE) branch to campus voice traffic Pass This test case will setup and verify the 2547oDMVPN (Hub used as a PE router) voice traffic can be successfully sent between branch-to-campus. Both the hub and the remote branch routers will be configured as PE routers in a PE-PE topology. The hub PE will also connect to the MPLS core in a PE-PE configure. Thus, the VPN defined in the large MPLS campus can be delivered to the branch via this solution. The DMVPN provided the flexible overlay model, plus the security the WAN solution needed. C7200 will be tested as the WAN hub, c7200 and c2800/c3800 will be tested as the branch routers. Both real phone (CCM and Cisco IP Phone) and simulated voice traffic are used. 2547oDMVPN(PE) branch to campus video Pass traffic This test case will setup and verify the 2547oDMVPN (Hub used as a PE router) video traffic can be successfully sent between branch-to-campus. Both the hub and the remote branch routers will be configured as PE routers in a PE-PE topology. The hub PE will also connect to the MPLS core in a PE-PE configure. Thus, the VPN defined in the large MPLS campus can be delivered to the branch via this solution. The DMVPN provided the flexible overlay model, plus the security the WAN solution needed. C7200 will be tested as the WAN hub, c7200 and c2800/c3800 will be tested as the branch routers. Multicast and mVPN is the technology to transport the IPTV multicast streams. IPTV is used as a real application to verify the solution. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II B-8 — Pass — — Pass — Appendix B Test Case Descriptions and Results B.3 2547oDMVPN (Hub as PE Role) Deployment Model 2547oDMVPN(PE) branch to campus mixed traffic Pass — Pass — This test case will setup and verify the 2547oDMVPN (Hub used as a PE router) mixed traffic can be successfully sent between branch-to-campus. Both the hub and the remote branch routers will be configured as PE routers in a PE-PE topology. The hub PE will also connect to the MPLS core in a PE-PE configure. Thus, the VPN defined in the large MPLS campus can be delivered to the branch via this solution. The DMVPN provided the flexible overlay model, plus the security the WAN solution needed. C7200 will be tested as the WAN hub, c7200 and c2800/c3800 will be tested as the branch routers. Both real application (Cisco CCM/IP phone, IPTV server/clients) and simulated voice/video traffic from Ixia will be used. Essentially this test case is a combination of the data/voice/video test cases to verify them can work together. 2547oDMVPN(PE) branch to branch data traffic Pass — Pass — This test case will setup and verify the 2547oDMVPN (Hub used as a PE router) data traffic can be successfully sent between branch-to-branch. The branch-to-branch communication is done via hub as recommended in the design guide (EDCS-536219). Both the hub and the remote branch routers will be configured as PE routers in a PE-PE topology. The hub PE will also connect to the MPLS core in a PE-PE configure. Thus, the VPN defined in the large MPLS campus can be delivered to the branch via this solution. The DMVPN provided the flexible overlay model, plus the security the WAN solution needed. C7200 will be tested as the WAN hub, c7200 and c2800/c3800 will be tested as the branch routers. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II B-9 Appendix B Test Case Descriptions and Results B.4 2547oDMVPN (Hub as P Role) Deployment Model 2547oDMVPN(PE) branch to branch voice traffic Pass This test case will setup and verify the 2547oDMVPN (Hub used as a PE router) voice traffic can be successfully sent between branch-to-branch. Both the hub and the remote branch routers will be configured as PE routers in a PE-PE topology. The hub PE will also connect to the MPLS core in a PE-PE configure. Thus, the VPN defined in the large MPLS campus can be delivered to the branch via this solution. The DMVPN provided the flexible overlay model, plus the security the WAN solution needed. C7200 will be tested as the WAN hub, c7200 and c2800/c3800 will be tested as the branch routers. Both real phone (CCM and Cisco IP Phone) and simulated voice traffic are used. 2547oDMVPN (Hub in P role): branch-to-branch Pass mixed traffic — Pass — — Pass — This test case will setup and verify the 2547oDMVPN (Hub used as a P router) mixed traffic can successfully be sent between branch-to-branch. The remote branch routers will be configured as PE router so the VPN defined in the large MPLS campus can be delivered to the branch via the 2547oDMVPN solution. The DMVPN provided the flexible overlay model, plus the security the WAN solution needed. C7200 will be tested as the WAN hub, c7200 and c2800/c3800 will be tested as the branch routers. HA is also provided by the redundant WAN hubs (hub11 and hub12) configuration. Both real application (Cisco CCM and IP phone) and simulated voice traffic from Ixia will be used. Essentially this test case is a combination of the data/voice test cases to verify them can work together. B.4 2547oDMVPN (Hub as P Role) Deployment Model Table B-4 Test System Integration Test Suites 2547oDMVPN (Hub as P Role) Deployment Model Manual Test Case Defects Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II B-10 Automation Test Case Defects Appendix B Test Case Descriptions and Results B.4 2547oDMVPN (Hub as P Role) Deployment Model 2547oDMVPN (Hub in P role): branch-to-campus baseline and data traffic Pass This test case will setup and verify the 2547oDMVPN (Hub used as a P router) data traffic can successfully be sent between branch-to-campus. The remote branch routers will be configured as PE router so the VPN defined in the large MPLS campus can be delivered to the branch via the 2547oDMVPN solution. The DMVPN provided the flexible overlay model, plus the security the WAN solution needed. C7200 will be tested as the WAN hub, c7200 and c2800/c3800 will be tested as the branch routers. HA is also provided by the redundant WAN hubs (hub11 and hub12) configuration. Hub 11 is the primary hub for all the branches. 2547oDMVPN (Hub in P role) branch-to-campus Pass Voice Traffic This test case will setup and verify the 2547oDMVPN (Hub used as a P router) voice traffic can successfully be sent between branch-to-campus. Both real phone (CCM and Cisco IP Phone) and simulated voice traffic are used. The remote branch routers will be configured as PE router so the VPN defined in the large MPLS campus can be delivered to the branch via the 2547oDMVPN solution. The DMVPN provided the flexible overlay model, plus the security the WAN solution needed. C7200 will be tested as the WAN hub, c7200 and c2800/c3800 will be tested as the branch routers. HA is also provided by the redundant WAN hubs (hub11 and hub12) configuration. 2547oDMVPN(P) branch-to-campus video Pass traffic — Pass — — Pass — — Pass — This test case will setup and verify the 2547oDMVPN (Hub used as a P router) video traffic can successfully be sent between branch-to-campus. The remote branch routers will be configured as PE router so the VPN defined in the large MPLS campus can be delivered to the branch via the 2547oDMVPN solution. The DMVPN provided the flexible overlay model, plus the security the WAN solution needed. C7200 will be tested as the WAN hub, c7200 and c2800/c3800 will be tested as the branch routers. HA is also provided by the redundant WAN hubs (hub11 and hub12) configuration. Multicast and mVPN is the technology to transport the IPTV multicast streams. IPTV is used as a real application to verify the solution. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II B-11 Appendix B Test Case Descriptions and Results B.4 2547oDMVPN (Hub as P Role) Deployment Model 2547oDMVPN(P) branch-to-campus mixed traffic Pass — Pass — This test case will setup and verify the 2547oDMVPN (Hub used as a P router) mixed traffic can successfully be sent between branch-to-campus. The remote branch routers will be configured as PE router so the VPN defined in the large MPLS campus can be delivered to the branch via the 2547oDMVPN solution. The DMVPN provided the flexible overlay model, plus the security the WAN solution needed. C7200 will be tested as the WAN hub, c7200 and c2800/c3800 will be tested as the branch routers. HA is also provided by the redundant WAN hubs (hub11 and hub12) configuration. Both real application (Cisco CCM/IP phone, IPTV server/clients) and simulated voice/video traffic from Ixia will be used. Essentially this test case is a combination of the data/voice/video test cases to verify them can work together. 2547oDMVPN(P) branch to branch data traffic Pass — Pass — — Pass — This test case covers data communications between branch and branch under MPLSoL2 solution model. Once MPLSoL2 baseline is up, data traffic will be verified between branch and branch. The test will focus on verifying: 1. Manually verify Data traffic between a Branch PC client to a PC server in another branch. 2.Verify QoS 3.Using test tool verify that data traffic between branch and branch meets the Service Level Requirements 2547oDMVPN (Hub in P role): branch-to-branch Pass Voice Traffic This test case will setup and verify the 2547oDMVPN (Hub used as a P router) voice traffic can successfully be sent between branch-to-branch. Both real phone (CCM and Cisco IP Phone) and simulated voice traffic are used. The remote branch routers will be configured as PE router so the VPN defined in the large MPLS campus can be delivered to the branch via the 2547oDMVPN solution. The DMVPN provided the flexible overlay model, plus the security the WAN solution needed. C7200 will be tested as the WAN hub, c7200 and c2800/c3800 will be tested as the branch routers. HA is also provided by the redundant WAN hubs (hub11 and hub12) configuration. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II B-12 Appendix B Test Case Descriptions and Results B.4 2547oDMVPN (Hub as P Role) Deployment Model 2547oDMVPN(P) branch to branch mixed traffic Pass — Pass — This test case will setup and verify the 2547oDMVPN (Hub used as a P router) mixed traffic can successfully be sent between branch-to-branch. The remote branch routers will be configured as PE router so the VPN defined in the large MPLS campus can be delivered to the branch via the 2547oDMVPN solution. The DMVPN provided the flexible overlay model, plus the security the WAN solution needed. C7200 will be tested as the WAN hub, c7200 and c2800/c3800 will be tested as the branch routers. HA is also provided by the redundant WAN hubs (hub11 and hub12) configuration. Both real application (Cisco CCM and IP phone) and simulated voice traffic from Ixia will be used. Essentially this test case is a combination of the data/voice test cases to verify them can work together. Scalability Test Suite 2547oDMVPN (Hub in P role) Hub Scalability Pass CSCsi79767 N/A N/A N/A N/A N/A N/A N/A N/A This test case will setup and verify the scalability of the 2547oDMVPN (Hub used as a P router) network. The network topology remains the same as described in the test case WM2-2547oDMVPN-SI-001-0001. Negative Test Suite 2547oDMVPN (Hub in P role): Failover Test Fail This test case will verify the network resiliency of a dual hub 2547oDMVPN (Hub used as a P router) topology when the primary hub or link goes down. The interruption of the traffic should be within the expected range. The same network topology and setup described in test case WM2-2547oDMVPN-NE-007-0001 will be used here. 2547oDMVPN(hub in P role): Hardware and Fail Control Plane Failure ) CSCsi50615 This failure was later found to be a misconfiguration CSCsi49487 This test case will verify the network resiliency of a dual hub 2547oDMVPN (Hub used as a P router) topology when the primary hub or link goes down. The interruption of the traffic should be within the expected range. The same network topology and setup described in test case WM2-2547oDMVPN-NE-007-0001 will be used here. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II B-13 Appendix B Test Case Descriptions and Results B.5 Inter-AS Deployment Model B.5 Inter-AS Deployment Model Table B-5 Test System Integration Test Suites Inter-AS : ASBR to ASBR with MP-eBGP: Baseline Test Inter-AS Deployment Model Manual Test Case Defects Automation Test Case Defects Pass — Pass — — Pass — — Pass — This test case will include all the configuration steps for the Inter-AS multihop eBGP option. A basic verification of all the features once they are configured is also part of the test case. Finally, an End-to-End connectivity test (from CE to CE) to verify the basic ip connection within a single VRF traversing across two MAN clouds. Inter-AS: ASBR to ASBR with MP-eBGP: Data Pass Traffic Verification This test case will verify communication between two PC clients across the two MAN clouds. The second part of the test case uses a traffic generator to send various data types to check QoS functionality across the MAN clouds. The following areas will be covered in this test case: Section A: PC/Linux Client Data traffic verification from MAN1 to MAN2 cloud Section B: QoS Data Traffic verification using State-full traffic generator Inter-AS: ASBR to ASBR with MP-eBGP: Voice Pass Verification This test case covers voice traffic verification across two MAN clouds. It includes manual voice verification using Cisco IP Phones. Test coverage includes verification of critical voice parameters like one-way latency, average one-way jitter and packet loss using either IxChariot or IxLoad. The same tool will be used to send Voice call signaling traffic. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II B-14 Appendix B Test Case Descriptions and Results B.5 Inter-AS Deployment Model Inter-AS : ASBR to ASBR with MP-eBGP: Video Pass Traffic Verification — Pass — — Pass — — N/A N/A N/A N/A — N/A N/A The purpose of this test case is to verify video traffic in a VRF traversing between two MAN clouds Test will focus on verifying: mVPN control plane verification over the Inter-AS links MAN1 to MAN2 Multicast control plane verification Manually verify Video reception using an IP TV viewer in MAN1 Campus receiving a multicast Verify unicast stream from IPTV server in Campus network across mVPN MPLS MAN Network. Using test tool verify that interactive Video and streaming video meets the Service Level Requirements. Inter-AS: ASBR to ASBR with MP-eBGP, Mixed Pass Traffic Test (Data, Voice and Video) This test case covers a mixture of all the data types running at the same type - Voice, video and Data traffic. The goal is to make sure that this specific data path can handle all 3 traffic types at the same time. Negative Test Suite Inter-AS: ASBR to ASBR with MP-eBGP, ASBR Pass Interface Failure The test case will test how the system reacts when one of the ASBR links to the SP core fails. There is a certain amount of "background" traffic that will load the system. The data traffic will be a mixture of simulated data streams that will load the ASBR's at approximate 50-60 % CPU and/or a link utilization of 50% of OC48 link. Inter-AS: ASBR to ASBR with MP-eBGP, ASBR Pass Reboot The test case will test how the system reacts when one of the ASBR reboots. There is a certain amount of "background" traffic that will load the system. The data traffic will be a mixture of simulated data streams that will load the ASBR's at approximate 50-60 % CPU and a link utilization of 50% of OC48 link. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II B-15 Appendix B Test Case Descriptions and Results B.5 Inter-AS Deployment Model Inter-AS: ASBR to ASBR with MP-eBGP, Clear BGP Routing Table Pass The test case will test how the system reacts when BGP routing tables are cleared from one of the ASBRs. There is a certain amount of "background" traffic that will load the system. The data traffic will be a mixture of simulated data streams that will load the ASBRs at approximate 50-60 % CPU and a link utilization of 50% of OC48 link. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II B-16 — N/A N/A A P P E N D I X C Defects Revised: October 23, 2007 C.1 CSCsi44003 MTU command disappear after reload Symptom: CLI (command line interface) "mtu 1508" under a subinterface disappears after a router reload. As a result, OSPF neighbor adjacency is not re-established due to MTU mismatch. Severity: Moderate Conditions: Cisco C3845 with VWIC-2MFT-T1-DI acting as a PE router in an MPLSoL2 deployment scenario. This failure does not occur on the same router with VWIC2-2MFT-T1/E1 Workaround: Use other T1 interface cards other than VWIC-2MFT-T1-DI Status: Active, ETA 9/7/07 C.2 CSCsj78913 Multicast VPN route flapping under traffic load Symptom: Multicast VPN route flap on a C2851 router Severity: Severe Conditions: Cisco 2851 router under a continuous multicast traffic. C2851 is acting as a PE router (Branch) in an MPLSoL2 WAN deployment scenario and with multicast VPN enabled. The problem is not observed in a C7200 that is also configured as a Branch PE router. Workaround: None Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II C-1 Appendix C Defects C.3 CSCek74416 Status: Unreproducible C.3 CSCek74416 DMVPN spoke crashed at crypto_ipsec_profile_map_val Symptom: One of the DMVPN spokes (C3845) experienced a software crash causing a router reload Severity: Severe Conditions: C3845 using 12.4(11)T1, although found originally using C3845, this defect is considered to affect any of the DMVPN spokes (ISR's and C7200). Large Scale DMVPN scalability testing with C7200 WAN hubs and 3 spokes (C3845, C2851 and c7200). Additonal DMVPN spokes are simulated making a total of 500 spokes. Workaround: None Status: Duplicate of CSCd73250, fix integrated in 12.4(16.13)T C.4 CSCsi50615 In 2547oDMVPN, OSPF does not run on DMVPN IPSec Tunnel after reload hub Symptom: Failure to restore OSPF neighbor adjacency after a redundant hub reload Severity: Severe Conditions: In a 2547oDMVPN deployment scenario where there are two redundant hubs. OSPF and LDP configured to run over DMVPN Workaround: In each spoke, enable and disable Ipsec in all the tunnels Status: Junk This is due to misconfiguration, the second tunnel does not come up because the mGRE spokes needs to have unique tunnel keys. C.5 CSCsi49487 2547oDMVPN: MDT BGP Peers can not be restored back after reload Hub Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II C-2 Appendix C Defects C.6 CSCsi79767 Symptom: When reloading DMVPN hub router, mVPN (multicast VPN) can not be restored back. Multicast streams can not flow from campus to branches. Severity: Severe Conditions: In 2547oDMVPN, Hub is configured as P role. LDP is running over DMVPN tunnel. mVPN is enabled in MPLS core. Workaround: "clear ip bgp vrf <vrf_name>" on campus PE's Status: Active C.6 CSCsi79767 NHRP network-ids can be different Symptom: Misleading use of network-id for NHRP configuration. This is only a documentation problem. Severity: Moderate Conditions: 2547oDMVPN with Hub acting as a P router. Workaround: None Status: New Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II C-3 Appendix C C.6 CSCsi79767 Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II C-4 Defects A P P E N D I X D Definition of Test Types Revised: October 23, 2007 • System Integration System Integration has two major components, feature combination and feature interaction. Feature combination focuses on testing a feature when various combinations of other features are enabled. Feature interaction test were conducted to verify dependencies between features. • Scalability Scalability testing measures the limit of a particular variable when all others are constant in a system level environment. For example, the number of routing entries that the system can support or number of OSPF neighbors. • Negative Negative testing concerns error handling and robustness. Erroneous inputs can be applied at the system level to verify behavior agains error handling specifications. Unspecified inputs or conditions, including fault injection, can be applied to asses the system level robustness. Redundancy Testing is placed under the negative test suite and it primarily pertains to testing network availability, e.g. validation of redundant WAN links. • Reliability System reliability is the probability that the system will work without failure for a specified period of time. Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II D-1 Appendix D Next Generation Enterprise MPLS-Based WAN Cisco Validated Design II D-2 Definition of Test Types