Where NetFlow and Packet Capture
Complement Each Other
June 17th, 2010
Michael Patterson
CEO | Plixer International, Inc.
SHARKFEST ‘10
Stanford University
June 14-17, 2010
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Course Outline
• What NetFlow is and how it works
• Egress or Ingress
• Comparison of the data exported by
NetFlow vs. Packet Analysis
• What’s next in NetFlow, where the
technology is going
• Summary
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
What is NetFlow?
How does it work?
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Voice Traffic
Database Traffic
Instant Messenger
Web Browsing
Private & Business Email
Video Conferencing
Music streaming
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
A
A - sending to B is one
flow entry on every
NetFlow capable
router / switch in the
path
B - acknowledging A is a
2nd flow
B
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Scrutinizer Accepts
•
•
•
•
NetFlow all Versions
sFlow version 2,4 and 5
IPFIX
NetStream
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
2 Flows per Connection
A
B
B
A
2
1
A
3
Router
B
4
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Who Supports NetFlow?
•
•
•
•
•
•
3Com
Adtran
Cisco
Enterasys
Expand
Juniper
•
•
•
•
•
•
Mikrotik
nProbe
Riverbed
VMWare
Vyatta
Others…
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
•
•
•
•
•
•
•
Cisco
Enterasys
Foundry
Hewlett Packard
Nortel
nProbe, nBox
Many More
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
MAC Addresses and VLAN IDs
• MAC addresses via Cisco ‘Flexible’ NetFlow
(aka NetFlow v9)
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
NetFlow or sFlow
• sFlow is an RFC not a standard
• Sampling of every N packets technology
– Can’t be used for IP accounting like NetFlow
• Maintained by Inmon
• Much less expensive for vendors to implement
• Vendors: 3Com, AlaxalA, Alcatel-Lucent, Allied Telesis,
Brocade, D-Link, Extreme Networks, Enterasys, Force10
Networks, H3C, Hewlett-Packard, Hitachi, Juniper Networks,
NEC and many others
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
NetFlow NBAR
• NBAR stands for Network Based Application
Recognition
• How many of you care if skype or pandora is
on your network? Perhaps you don’t mind it
but, want to know how much there is. Well,
NBAR helps us with deeper packet inspection
that isn’t available with traditional NetFlow.
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Router CPU Impact
• Typically, the impact on the router’s CPU is
negligible.
• However, NetFlow NBAR can clobber some
routers.
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Egress or Ingress
• Most of us are exporting NetFlow v5 which only supports
ingress NetFlow. This means that traffic coming in on an
interface is monitored and exported in NetFlow datagrams.
• Most NetFlow vendors look at where an ingress flow is headed
by looking at the destination interface. Using this information,
we can determine outbound utilization on any given interface
as long as AND THIS IS IMPORTANT, you enable NetFlow v5 on
all interfaces of the switch or router.
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
When to use Egress
• In WAN compression environments (e.g. Cisco WAAS, Riverbed,
etc.), we need to see traffic after it was compressed. Using Ingress
flows causes an over stated outbound utilization on the WAN
interface. Egress flows are calculated after compression.
• In multicast environments, ingress multicast flows have a
destination interface of 0 because the router doesn’t know what
interface they will go out until after it processes the
datagrams. Exporting egress flows delivers the destination
interface and as a result multiple flows are exported if the flow is
headed for multiple interfaces.
• When exporting NetFlow on only one interface of the router or
switch. Enabling both on a single interface means that all traffic in
and out is exported in NetFlow datagrams.
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Demonstration
Scrutinizer NetFlow & sFlow Analyzer
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
NetFlow and Packet
Analysis?
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Example 1: FTP Comparison
Steps for the Lab
• I started WireShark
• I logged in and FTP’d a file
• I logged out
• I stopped WireShark
• 6 Ingress Flows represent
2221 packets
• 6 Egress Flows represent
1123 packets
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Ingress
Lets count packets and compare with
Wireshark
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Displaying Ingress
Total = 2221 packets
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Displaying Ingress
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Egress
Lets count packets and compare with
Wireshark
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Displaying Ingress
Total = 1123 packets
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Displaying Egress
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Capture Details
Lets compare NetFlow details to
Packet details
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
What about Flags?
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Example 2: www.llbean.com
Steps for the Lab
• I started WireShark
• I surfed to www.llbean.com
• I went to another web site
• I stopped WireShark
• 2 Ingress Flows represents 11
packets going out from my PC
• 1 Ingress Flow represents 13
packets coming back from
llbean.com
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Cisco
Router
11 packets
From my PC (10.1.7.5) NAT’d
by the firewall (66.186.184.62)
2 flows
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Enterasys
Switch
11 packets
From my PC (10.1.7.5)
On the Enterasys switch
before the router.
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
From www.llbean.com
13 packets
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
From www.llbean.com
13 packets
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Example 3: VoIP
Steps for the Lab
• I started WireShark
• I started iaxLite
• I made a call
• The other end picked up
• I hung up
• I closed iaxLite
• I stopped WireShark
• 1 Ingress Flow represents 1364
UDP packets
• 1 Egress Flow represents 1364
UDP packets
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
My Computer to the PBX
1364 packets
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
My Computer to the PBX
1364 packets
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
PBX to My Computer
1364 packets
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
PBX to My Computer
1364 packets
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Distributed Collectors
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Detecting Malware
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Network Behavior Analysis
• Network Behavior Analysis
– Constantly monitor NetFlow
and sFlow from selected
routers and switches
– Looks for traffic patterns
defined in behavioral
algorithms
– Additional filters can be
created to look for unique
circumstances
• Demonstration
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Future of NetFlow
Current Innovations
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Latency via NetFlow
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
RTT and Server Latency
These fields got cut.
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
URL Information
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
WAN Optimization Sizing
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Procflow from Gerald Combs
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
What is next from NetFlow?
•
•
•
•
Packet captures
Sampling Flows
IPv6 is here and we are reporting on it.
Syslogs: Cisco ASA. We already provide
reports on this.
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010
Summary
•
•
•
•
Ingress Vs. Egress NetFlow
Advanced Filtering to narrow in on problems
How and When to leverage reports
The differences between NetFlow and Packet
Capture
• Where the technology is going
SHARKFEST ‘10 | Stanford University | June 14 –17, 2010