Chap 8 – Network Troubleshooting Learning Objectives •Establish a network baseline •Describe troubleshooting methodologies and troubleshooting tools •Describe the common issues that occur during WAN implementation •Troubleshoot enterprise network implementation issues 1 Chapter 8 Network Documentation Network Configuration Documentation End-System Documentation 2 Chapter 8 Network Documentation Process When you document your network, you may have to gather information directly from routers and switches. Commands that are useful to the network documentation process include: • • • • • The ping command is used to test connectivity with neighboring devices before logging in to them. Pinging to other PCs in the network also initiates the MAC address auto-discovery process. The telnet command is used to log in remotely to a device for accessing configuration information. The show ip interface brief command is used to display the up or down status and IP address of all interfaces on a device. The show ip route command is used to display the routing table in a router to learn the directly connected neighbors, more remote devices (through learned routes), and the routing protocols that have been configured. The show cdp neighbor detail command is used to obtain detailed information about directly connected Cisco neighbor devices. 3 Chapter 8 Establish a Network Baseline A Network Baseline helps to determine the "personality" of the network and provides answers to the following questions: 1. How does the network perform during a normal or average day? 2. Where are the under-utilised and over-utilised areas? 3. Where are the most errors occurring? 4. What thresholds should be set for the devices that need to be monitored? 5. Can the network deliver the identified policies? 4 Chapter 8 Establishing a Baseline 1. Determine what types of data to collect 2. Identify devices and ports of interest 3. Determine the baseline duration 5 Chapter 8 Establishing a Baseline Sophisticated network management software is often used to baseline large and complex networks. 6 Chapter 8 Systematic Troubleshooting •It is important to analyse the network as a whole rather than in a piecemeal fashion. •A systematic approach minimises confusion and cuts down on time otherwise wasted with trial and error. •Logical networking models, such as the OSI and TCP/IP models, separate network functionality into modular layers. When troubleshooting, these layered models can be applied to the physical network to isolate network problems. 7 Chapter 8 Troubleshooting Methodologies and Tools Stage 1 Stage 2 Stage 3 Gather Symptoms Isolate Problem Correct Problem Retest – Further Problems? 8 Chapter 8 Gathering Symptoms • • • • • Step 1. Analyse existing symptoms – Read fault report and question user to understand problem. Step 2. Determine ownership – Is the problem within your system, or those of an external agency? Step 3. Narrow the scope - Determine if the problem is at the core, distribution, or access layer of the network. At the identified layer, analyse the existing symptoms and use your knowledge of the network topology to determine which pieces of equipment are the most likely cause. Step 4. Gather symptoms from suspect devices - Using a layered troubleshooting approach, gather hardware and software symptoms from the suspect devices. Step 5. Document symptoms - Sometimes the problem can be solved using the documented symptoms. 9 Chapter 8 Gathering Symptoms 10 Chapter 8 Software Troubleshooting Tools •Network management system (NMS) tools include device-level monitoring, configuration, and fault management tools. •On-line network device vendor knowledge bases have become indispensable sources of information. •A protocol analyser decodes the various protocol layers in a recorded frame and presents this information in a relatively easy to use format. 11 Chapter 8 Hardware Troubleshooting Tools Digital Multimeter (DMM) Cable Tester 12 Cable Analyser Network Analyser Chapter 8 Steps In WAN Design Locate LANs Analyse Traffic Plan Topology Plan Bandwidth Choose Technology Cost & Evaluate 13 Where are they? How many users? Video? Data? Voice? Latency/Jitter Circuit? Packet? Cell? UTP? Fibre? Switches? Routers? Can you afford it? Chapter 8 WAN Traffic Analysis •Determine what data traffic must be carried, its origin, and its destination. •WANs carry a variety of traffic types with varying requirements for bandwidth, latency, and jitter: 14 Chapter 8 Plan the WAN Topology •When many locations must be joined, a hierarchical solution is recommended. •LANs can be grouped into areas and interconnected them to form a region, which are in turn interconnected to form the core of the WAN. •The area would have a star topology, with the hubs of the stars linked to form the region. •Regions could be geographic, connecting between three and 10 areas, and the hub of each region could be linked pointto-point. 15 Chapter 8 WAN Technology Bandwidth •A typical private WAN uses a combination of technologies that are usually chosen based on traffic type and volume. 16 Chapter 8 Symptoms of Physical Layer Problems • • • • • • 17 Performance Lower than baseline Loss of connectivity High collisions Network bottlenecks or congestion Nigh CPU utilisation rates Console error messages Chapter 8 Causes of Physical Layer Problems • Power related problems Hardware faults • Cabling faults • • • • • • 18 Attenuation Noise (EMI) Interface configuration errors Exceeding design limits CPU overload Chapter 8 Physical Layer Troubleshooting Check for bad cables or connections Check that correct cable standard has been used Check if devices have been cabled correctly Cable Analyser Verify correct interface configuration Check operating statistics and data error rates 19 Chapter 8 Physical Layer Troubleshooting show interface serial ## 20 Chapter 8 Physical Layer Troubleshooting 21 Chapter 8 Symptoms of Data Link Layer Problems • • • • • 22 No connectivity at the network layer or above No functionality at the network layer or above Network performance below baseline Excessive broadcasts Console error messages Hardware faults Chapter 8 Causes of Data Link Layer Problems • • • • 23 Encapsulation errors Address mapping errors Framing errors STP failures or loops Chapter 8 Data Link Layer Troubleshooting Check encapsulation settings Check authentication settings Check LMI configuration (frame relay) Check PVC status (frame relay) Check DLCI mapping (frame relay) 24 Chapter 8 Data Link Layer Troubleshooting 1. Identify that an STP loop is occurring - Loss of connectivity, high CPU utilization on routers connected to affected segments or VLANs , high link utilization (often 100 percent) 2. Discover the scope of the loop - highest priority is to stop the loop and restore network operation by discovering which ports are involved. 3. Break the loop - shut down or disconnect the involved ports one at a time. 4. Find and fix the cause of the loop - investigate the topology diagram to find a redundant path. 5. Restore the redundancy - after the device or link that is causing the loop has been found and the problem has been resolved, restore the redundant links that were disconnected. 25 Chapter 8 Symptoms of Network Layer Problems • • 26 Network failure Network performance below baseline Chapter 8 Causes of Network Layer Problems • • • • 27 Topology changes Connectivity problems Neighbor issues Routing issue Chapter 8 Network Layer Troubleshooting Check for network topology changes Check for equipment & connectivity problems Check routing neighbor relationships Check topology tables Check routing tables 28 Chapter 8 Symptoms of Transport Layer Problems • • • • 29 Intermittent network problems Security problems Address translation problems Problems with specific traffic types Chapter 8 Causes of Transport Layer ACL Problems • • • • • • • • 30 Applied to incorrect traffic Incorrect order Implicit deny any any Addresses & wildcard masks TCP / UDP selection Source & Destination ports Use of established keyword Uncommon protocols Chapter 8 Causes of Transport Layer NAT Problems • • • 31 Interoperability issues Incorrect static NAT Incorrect NAT timers Chapter 8 Symptoms of Application Layer Problems • • • • 32 User complaints about slow performance Console error messages System log file messages Network management system alarms Chapter 8 Application Layer Troubleshooting Ping the default gateway Verify end-to-end connectivity Verify ACL and NAT operation Troubleshoot upper layer protocol connectivity 33 Chapter 8 Chap 8 – Network Troubleshooting Learning Objectives •Establish a network baseline •Describe troubleshooting methodologies and troubleshooting tools •Describe the common issues that occur during WAN implementation •Troubleshoot enterprise network implementation issues 34 Chapter 8 Any Questions? 35 Chapter 8