How ReporterAnalyzer and NetFlow Display Tunnel
Traffic
Logical Interfaces
In “NetFlow Deployment on Logical Interfaces”, Cisco describes how NetFlow can be
deployed on various types of logical interfaces, such as Frame Relay and ATM
subinterfaces, and L2TP and GRE virtual interfaces. This document focuses on GRE
tunnels and how ReporterAnalyzer displays the data received from a NetFlow enabled
GRE tunnel.
The Cisco document describes a special consideration for NetFlow enabled tunnels – “Be
careful about counting same packet twice”. Cisco uses the ‘sh ip cache flow’ command
to illustrate this fact. This document will illustrate this point using ReporterAnalyzer.
Lab Setup
For the test, I used two Cisco 2500 routers, each having two Ethernet interfaces. The
routers are directly connected using one of their interface and a tunnel is defined between
the them. A simple setup - what goes in goes out. A file transfer of 441 MB
(462,856,192 bytes) was performed from 192.168.0.90 to 192.168.99.22 – so that the file
traversed each of the routers and passed though the tunnel. The file transfer used tcp445, ms-ds.
SELAB01(192.168.0.45)::Ethernet0 – NetQoS Internal Network
SELAB01::Eth0 directly connects to the 192.168.0.0 subnet. During the test transfer this
was the first interface encountered. Notice the ‘In’ volume for ms-ds of 473MB
correlates to the file size of 441MB, the ‘Out’ volume is only 8.88MB, and no GRE
traffic is found.
SELAB02(192.168.99.254)::Ethernet1 – Sales Engineering Network
SELAB02::Eth1 directly connects to the 192.168.99.0 subnet. This is the final interface
along the path and no tunnel is defined. The ‘In’ traffic of 8.88MB correlates exactly to
the ‘Out’ traffic for SELAB01::Eth0, and the ‘Out’ traffic of 473MB correlates to the ‘In’
traffic of SELAB01::Eth0. No GRE traffic is found.
SELAB01(192.168.0.45)::Tunnel0 – Tunnel to SELAB02
After entering SELAB01::Eth0, the packets are sent to SELAB01::Tunnel0. And here is
where we see the affects of “duplicate” flows. First, notice the ‘Out’ traffic of 478.38MB
of ms-ds. This makes perfect sense. What goes in SELAB01 goes out the tunnel towards
SELAB02. Also, the ‘In’ traffic of 8.88MB correlates exactly to the ‘In’ traffic of
SELAB02:Eth1. Again, what goes in SELAB02:Eth1 will go in SELAB01::Tunnel0.
But, if you haven’t already noticed, check out the 487.76MB of GRE “into” this
interface. Where does it come from?
Look at the following out of ‘sh ip cache flow’ for SELAB01:
SrcIf
Et1
Tu0
Et0
Tu0
Et0
SrcIPaddress
10.2.0.2
10.2.0.1
192.168.0.90
192.168.99.22
192.168.0.90
DstIf
Local
Et1
Tu0
Et0
Local
DstIPaddress
10.2.0.1
10.2.0.2
192.168.99.22
192.168.0.90
192.168.0.45
Pr SrcP DstP Pkts
2F 0000 0000 7880
2F 0000 0000 15K
06 0CAC 01BD 15K
06 01BD 0CAC 7880
06 0DDA 0017 7
The second line shows that the tunnel is the source (ingress) interface for a GRE flow to
Ethernet1 of SELAB01. The source address of 10.2.0.1 is the IP address for
SELAB01::Ethernet1. The destination address of 10.2.0.2 is the IP address of the
physical interface of other end of the tunnel on SELAB02::Ethernet0. In
ReporterAnalyzer, the conversation associated with this ‘In’ GRE traffic for
SELAB01:tunnel0 is 10.2.0.1->10.2.0.2.
So, during the transfer from 192.168.0.90 to 192.168.99.22, a flow is created when traffic
arrives at Eth0 of SELAB01 with an egress interface of Tunnel0, and we derive the ‘Out’
traffic via Out-Calculation. However, during GRE encapsulation, another flow is created
with Tunnel0 as the source and Eth1, the tunnel’s physical interface, as the destination.
Thus, there is GRE traffic ‘In’ to the tunnel. The same phenomenon occurs with
SELAB02.
However, when I enabled CEF on SELAB02, this “duplicate” GRE traffic stopped
showing up.
SrcIf
Et0
Et0
Tu0
Et1
SrcIPaddress
DstIf
10.2.0.1
Local
10.2.0.1
Local
192.168.0.90
Et1
192.168.99.22 Tu0
DstIPaddress
Pr
10.2.0.2
2F
10.2.0.2
2F
192.168.99.22 06
192.168.0.90
06
SrcP DstP
0000 0000
0000 0000
0CAC 01BD
01BD 0CAC
Pkts
15K
20
13K
6762
Apparently, with CEF, the GRE encapsulation traffic is associated with Ethernet0 and the
router itself (Local). Thus in ReporterAnalyzer, ‘In’ traffic is would be represented on
the physical interface of the tunnel, not the tunnel itself, and ‘Out’ would be found on the
Aggr(BC/MC) interface instead of the physical interface of the tunnel. Confusing
enough?
Router Configuration
SELAB01
ip flow-cache timeout inactive 10
ip flow-cache timeout active 1
!
!
interface Tunnel0
description Tunnel to SELAB02
bandwidth 10000
ip address 10.3.0.2 255.255.0.0
ip route-cache flow
tunnel source Ethernet1
tunnel destination 10.2.0.2
!
interface Ethernet0
description NetQoS Internal Network
ip address 192.168.0.45 255.255.255.0
ip route-cache flow
!
interface Ethernet1
description To SELAB02 (GRE Only)
ip address 10.2.0.1 255.255.0.0
ip route-cache flow
!
interface Serial0
no ip address
no ip mroute-cache
shutdown
!
interface Serial1
no ip address
no ip mroute-cache
shutdown
!
ip flow-export source Ethernet0
ip flow-export version 5
ip flow-export destination 192.168.99.212 9995
ip classless
ip route 0.0.0.0 0.0.0.0 Ethernet0
ip route 192.168.99.0 255.255.255.0 Tunnel0
ip route 192.168.100.0 255.255.255.0 192.168.0.250
SELAB02
!
ip subnet-zero
ip flow-cache timeout inactive 10
ip flow-cache timeout active 1
!
interface Tunnel0
description Tunnel to SELAB01
bandwidth 10000
ip address 10.3.0.1 255.255.0.0
ip route-cache flow
tunnel source Ethernet0
tunnel destination 10.2.0.1
!
interface Ethernet0
description To SELAB01 (GRE Only)
ip address 10.2.0.2 255.255.0.0
ip route-cache flow
!
interface Ethernet1
description Sales Engineering Network
ip address 192.168.99.254 255.255.255.0
ip route-cache flow
!
interface Serial0
no ip address
shutdown
!
interface Serial1
no ip address
shutdown
!
ip flow-export source Ethernet1
ip flow-export version 5
ip flow-export destination 192.168.99.212 9995