Application and Network Monitoring
Lorna Robertshaw, Director of Applications Engineering
OPNET Technologies
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
About OPNET Technologies, Inc.®
Corporate Overview
•
•
•
•
•
Founded in 1986
Publicly traded (NASDAQ: OPNT)
HQ in Bethesda, MD
Approximately 600 employees
Worldwide presence through direct offices and channel partners
Best-in-Class Solutions and Services
• Application Performance Management
• Network Engineering, Operations, and Planning
• Network R&D
Strong Financial Track Record
• Long history of profitability
• Trailing 12-month revenue of over $120M
• Approximately 25% of revenue re-invested in R&D
Broad Customer Base
•
•
•
•
Corporate Enterprises
Government Agencies/DoD
Service Providers
Network Equipment Manufacturers
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
OPNET Solutions Portfolio
Application Performance
Management (APM)
Network Engineering,
Operations, and Planning
Analytics for Networked Applications
Network Planning and Engineering for Enterprises
End-User Experience Monitoring & Real-Time Network Analytics
Network Planning and Engineering for Service Providers
Real-Time Application Monitoring and Analytics
Transport Network Planning and Engineering
Systems Capacity Planning for Enterprises
Network Audit, Security, and Policy Compliance
Automated Up-to-Date Network Diagramming
Network R&D
Modeling and Simulation for Defense Communications
Wireless Network Modeling and Simulation
Accelerating Network R&D
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Agenda
• Monitoring Application Behavior
–Case Study: Impact of rogue application and users
–Case Study: Impact of worms and viruses
–Case Study: Impact of bottlenecks
–Monitoring, Triage, and Forensics
–Monitoring network and application behavior with OPNET ACE Live
–Deep-dive packet analysis and forensics with ACE Analyst
–Using application characterizations in OPNET Modeler
• Auditing Network Configuration
–Case Study: Impact of misconfigurations on WAN infrastructure
–Case Study: Default passwords on Internet-facing routers
–Auditing device configurations with Sentinel
–Providing network diagramming through NetMapper
• Questions
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Monitoring Application Behavior
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Case Study: Impact of Rogue Applications
•
•
•
•
Company that does scientific research for defense agencies
Large monthly costs for WAN connection between two main sites
Link is often near saturation, so cost is justified
Investigation finds one user responsible for 1/3 of total inbound traffic throughout
workday – syncing home computer to work computer
• Possible security threat
• Huge monthly expense to company
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Case Study: Impact of Worms and Viruses
• The perfect storm: Large software company. Battles
between IT staff and developers over management
of development servers.
• Blaster Worm (August 2003)
• Worm caused infected computers to become
unstable
• Infected computers also caused major network
outages that impacted non-infected computers!
• Network was unusable but no one knew why
• Application monitoring showed ~150 infected
machines sending ARP requests for every IP they
could think of
• It took 5 hours to find and unplug infected computers
• Major business impact – tech support was down,
customer support site was down, lost
productivity
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Case Study: Impact of Network Bottlenecks
• Medical Service Provider
• One data center with large research facilities (high bandwidth),
•
•
•
•
•
hospitals (lower bandwidth), and small strategic sites (T1,
sufficient for 3-4 users)
Citrix, Terminal services, WAN Optimizers deployed throughout
to overcome network latency issues
Tricky environment to troubleshoot and gain visibility!
Users in low bandwidth locations experience high network
congestion and retransmissions
Monitoring showed that congestion correlated with times users
were printing
Single print server in the Data Center was a huge bottleneck
and was impacting high priority traffic to the strategic sites
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Three Dimensions of Application Performance
Management
Monitoring: high-level view
•
•
•
•
•
Broad visibility (network, server…)
Real-time dashboards
Alerts when user experience degrades
SLA violations
Trending and historical data
Triage: initial troubleshooting
•
•
•
•
Localize problem (who, what, when, how bad)
Due to network or server?
Which team to call next?
Snapshot and archive forensic data
Forensics: root cause
• Follow user transaction across network and
through servers
• Identify specific cause (network event, line of
code, etc.)
9
OPNET
Confidential
– Not
release
third
parties.©©2009
2009OPNET
OPNETTechnologies,
Technologies,Inc.
Inc.AllAllrights
rightsreserved.
reserved.OPNET
OPNETand
andOPNET
OPNETproduct
productnames
namesare
aretrademarks
trademarksofofOPNET
OPNETTechnologies,
Technologies,
Inc.
OPNET
Confidential
– Not
forfor
release
to to
third
parties.
Inc.
trademarks
property
their
respective
owners
and
used
herein
identification
purposes
only.
AllAll
trademarks
areare
thethe
property
of of
their
respective
owners
and
areare
used
herein
forfor
identification
purposes
only.
• Real-time agentless performance monitoring
• Broad coverage with a small footprint (all users and all
applications)
• Localize performance problems and differentiate between
network and server delay
• Snapshot detailed data for forensic analysis
ACE Live
ACE Live
Data Center
10
OPNET
Confidential
– Not
release
third
parties.©©2009
2009OPNET
OPNETTechnologies,
Technologies,Inc.
Inc.AllAllrights
rightsreserved.
reserved.OPNET
OPNETand
andOPNET
OPNETproduct
productnames
namesare
aretrademarks
trademarksofofOPNET
OPNETTechnologies,
Technologies,
Inc.
OPNET
Confidential
– Not
forfor
release
to to
third
parties.
Inc.
trademarks
property
their
respective
owners
and
used
herein
identification
purposes
only.
AllAll
trademarks
areare
thethe
property
of of
their
respective
owners
and
areare
used
herein
forfor
identification
purposes
only.
End User Experience Monitoring
• 24x7 application monitoring appliance
– End-user response time for all transactions and users
• Auto-discovers applications out-of-the-box
Executive dashboard of real-time performance
– Oracle, Peoplesoft, SAP, Microsoft, IM, P2P, others
• Intuitive, easy-to-use, low TCO
– One-click guided work flows
– Web-based dashboards; customizable reports
– Installed and configured within 1 hour
• Unified views across the enterprise
• Automatic analysis
– Components of delay, top-talkers
– Dynamic thresholds— “learns” abnormal behavior
– Historical trending (up to one year)
• Real-time VoIP performance management
• NetFlow collection
Quick, easy network troubleshooting
– NetFlow and user response time in a unified view in a
single appliance
• Exclusive: Integrated monitoring and
troubleshooting
– Integrates with ACE Analyst for root cause analysis
SLA monitor highlights poor performance
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
ACE Live “Insights”
• Easy guided workflows for troubleshooting and analysis
–Point-and-click wizards automate best practices
–Accomplish complex tasks at a mouse-click
–Customizable
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Bandwidth Hogs
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Alerts: Potential DoS Attacks
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Worm Hunt: Detect External Attacks
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
End-User Response Times: Server Delay
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
End-User Response Times: Network Delays
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Analytics for Networked Applications
• Automatic root-cause analysis
– Visualize application behavior across the network
– Diagnose root causes of response-time delay
– Validate proposed solutions
– Certify new applications prior to rollout
• Restores network-tier visibility in WANoptimized environments
– Support for leading vendors (e.g. Riverbed, Cisco,
Juniper)
• Response time prediction using a behavioral
Summarize components of response-time delay
application model
– New application deployment
– Data center migrations
– Server consolidation and virtualization
– WAN optimization deployment
– Application deployment to new locations
• Over 700 protocol and application decodes
– Citrix, Oracle, SQL Server, Web Services, others
Predict response times
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
ACE Analyst for Deep Dive Forensics
•
•
•
•
Visually see the connections
Gantt chart of each conversation
Drill into packet decodes
Shorten time/skillset needed to analyze packet captures
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Application Characterization for
simulation in OPNET Modeler
• Real traffic patterns add accuracy to simulated models
• Simulate DoS attacks etc.
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Application Monitoring: Summary
• Quality monitoring tools will help you:
–Weed out rogue applications
–Detect and study security threats
–Only pay for bandwidth you need
–Avoid congestion caused by inefficient architecture
–Understand import of issues on end-user experience
–TRIAGE problems and allow deeper dive into FORENSICS tools
• Keys to deploying application monitoring solutions:
–Diverse user community with different access levels, cross-disciplinary communication
–User training
–Hook into existing tools wherever possible, look for integrated tool suites rather than
point solutions
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Network Configuration Monitoring
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Case Study: Impact of misconfigurations
on WAN infrastructure
• Global ISP
• Core routers have HUGE routing tables
• Peering points to customer networks use route filters to avoid bombarding CE
routers with Internet routing tables
• Operator fat fingers route filter name
• Cisco IOS responds by sharing no routes
• Months pass…
•
•
•
•
IOS upgrade occurs
IOS throws out the command altogether
ALL routes sent to CE router
Outage in middle of business day
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Case Study: Default Passwords
• Large insurance company with stringent regulatory requirements
(SOX, HIPAA)
• Some routers and switches in production network still have “staging”
configurations
• Default username/pw combinations (cisco/test etc) found on Internet
facing devices
• Production community strings found on devices
• Major changes required to entire network in case the devices had
been compromised
• Could have been worse!
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Network Audit, Security, and Policy-Compliance
• Reduce network outages
–Detect configuration problems before they
disrupt network operations
–Automatically audit production network
configuration with ~750 rules
• Ensure network security
–200+ security rules
• Demonstrate regulatory compliance
–Generate self-documenting, customizable
reports
–Leverage rule templates for rapid
customization
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Sentinel Architecture
Production Network
Scheduled
Audit Engine
Configuration
& Topology
Third Party Data Sources
Near Real-Time
Comprehensive
Network Model
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Security Standards and Guidelines
Standard/Guide
PCI Data Security
Standard
Description
Describes the Payment Card Industry (PCI) Data Security Standard (DSS) requirements.
Applicable Organizations
* Banks
* Credit Card Merchants
PCI DSS requirements are applicable if a Primary Account Number (PAN) is stored, processed, or
transmitted.
NIST Special Publication
800-53
(also basis for FISMA
compliance)
Provides technical guidance to enhance the confidentiality, integrity, and availability of Federal
Information Systems.
DISA Network
Infrastructure STIG
Provides security configuration guidance to enhance the confidentiality, integrity, and availability of
sensitive DoD Automated Information Systems (AISs).
* DoD
* Defense Contractors
* Federal Agencies
This document is provided by NIST as part of its statutory responsibilities under the Federal Information
Security Management Act (FISMA) of 2002, P.L. 107-347.
* DoD
* Federal Agencies
* Defense Contractors
This Security Technical Implementation Guide (STIG) is provided under the authority of DoD Directive
8500.1.
NSA Router Security
Configuration Guide
Provides technical recommendations intended to help network administrators improve the security of
their routed networks.
The initial goal for this guide is to improve the security of the routers used on US Government
operational networks.
NSA Cisco IOS Switch
Security Configuration
Guide
Provides technical recommendations intended to help network administrators improve the security of
their switched networks.
* Federal Agencies
* DoD
* Enterprises
* Service Providers
* DoD
* Enterprises
* Service Providers
The initial goal for this guide is to improve the security of the switches used on DoD operational
networks.
Cisco SAFE Blueprint for
Enterprise Networks
Provides Cisco’s best practices to network administrators on designing and implementing secure
networks.
* Enterprises
ISO-17799
Provides guidelines and general principles for initiating, implementing, maintaining, and improving
information security in an organization.
* Enterprises
This is an International Standard developed by the International Organization for Standardization (ISO)
and the International Electro technical Commission (IEC).
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Example Sentinel Reports
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Example Sentinel Reports
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Automated Network Diagramming
• Automatically generate up-to-date
network diagrams
• Published in Microsoft Visio® format
• Comprehensive and detailed unified
network views
–Physical layouts
–Detailed configuration information
–Logical views including Layer 2/3,
VPN, OSPF, BGP, and VLANs
–Custom annotations
• Benefits
–Meet regulatory compliance
requirements: PCI, SOX, etc.
–Accelerate network troubleshooting
–Perform effective asset & change
management
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.
Questions?
OPNET Confidential – Not for release to third parties. © 2009 OPNET Technologies, Inc. All rights reserved. OPNET and OPNET product names are trademarks of OPNET Technologies, Inc.
All trademarks are the property of their respective owners and are used herein for identification purposes only.