ITU-T Workshop on
IP Traffic Flow Measurement
(Geneva, Switzerland, 24 March 2011)
Overview of IETF work on IP traffic flow measurement and current developments
Dr. Jürgen Quittek
General Manager
Network Research Division, NEC Europe Ltd.
Heidelberg, Germany
Geneva, 24 March 2011

IP packets and flows
Groups of IP packets sharing common characteristics
(e.g IP src/dst address, TOS field, protocol,
…
…
…
…
… or have a limited lifetime...
… and packets may belong to more than one flow
Typical reported flow information:
•start time
•end time
•#packets
•#bytes
Periodically reported for long lasting flows t
2

The general (passive) IP traffic measurement process packets
Packet
Capturing
PAYLOAD HEAD
PAYLOAD HEAD
PAYLOAD HEAD
PAYLOAD HEAD
Observation
Point
(router, probe, etc.)
Sampling packets
Filtering packets
Metering process both steps may be trivial (1:1 sampling, no filtering) packet reports
Classification &
Flow Recording flow records
Exporting process flow records
Sampling flow records
Filtering flow records
3

…
…
Meter:
Filters packets, timestamps them and associates
Pkts to flow(s)
The flow monitoring process
…
…
Flow cache:
Creates/Removes/Updates flow records
• Flow Key
• Flow start time
• Flow last update time
• # Pkts
• # Bytes
• ….
• ….
Exp HD
IETF IPFIX
(Netflow v9) info info info
Database
Router functionality or dedicated Probe
Exporter:
Reads Flow cache, prepares and sends export packets
Exp HD info info info
Collector:
Receives export packets, interfaces to applications
4

Flow monitoring issues
Flows have very different characteristics long-/short-lived, high/low volume, etc.
Creating/updating flow record at high speed links packet sampling fast memory for flow cache, flow sampling
Timing out flows ( TCP FIN/RST vs. timeout )
Reporting flow cache reading effort, reporting frequency selective report
Reporting format fixed format: Netflow 5 template based: Netflow 9, IPFIX
5

IETF activities on IP traffic measurement
Three working groups
IPPM: IP Performance Metrics defines metrics for performance measurements (delay, roundtrip time, loss, etc.)
IPFIX: IP Flow Information eXport defines protocol for export of flow data
PSAMP: Packet Sampling (concluded) defines protocol for export of packet data based on IPFIX
6

IPFIX protocol
IP Flow Information eXport
Established 2001
Main goal: Develop common IP traffic flow reporting protocol to be available on most future routers meeting requirements of many applications low hardware/software costs simple,
Scalable extensible http://datatracker.ietf.org/wg/ipfix/
7

Further requirements for IPFIX I
Distinguishing flows by
5-tuple (IP addresses, protocol, port)
MPLS label, TOS fields interface & direction
Flexible aggregation of flows
Metering Process timestamps flow timeouts
8

Further requirements for IPFIX II
Extensible information/data model flow properties and statistics many header fields anonymization
Reliable and secure data transfer congestion awareness push model reporting
Configuration
9

IPFIX architecture
Flow Information
Export
Exporting
Process
Metering
Process
Flow
Record
Collecting
Process
Application
PAYLOAD HEAD
PAYLOAD HEAD
PAYLOAD HEAD
PAYLOAD HEAD
PAYLOAD HEAD
PAYLOAD HEAD
PAYLOAD HEAD
PAYLOAD HEAD
Observation Point
10

Probe
E
M
O
Protocol
Converter
E
(Meter MIB)
M
O
Simple
Router
E
O
M
IPFIX devices
Complex
Router
O
M
E
O
M
Multiple
Exporters
E
O
M
O
E
M
M
M
O
E
M
Concentrator
C M E
Proxy
C E
O
M
E
C
Observation Point
Metering Process
Exporting Process
Collecting Process
11

IPFIX protocol design
Based on NetFlow version 9
Binary-coded flow record arrays
Templates for flow record formats first send a template then send data records with the format defined by the template
Runs over SCTP, TCP, UDP
12

IPFIX information model
A flow record contains header fields (transport, IP, sub-IP)
"flow keys" used for distinguishing flows counters for packets, bytes, etc.
time stamps further flow properties min/max values, duration, direction next hop IP address
BGP source AS, destination AS, next hop AS may also be used as flow keys
All defined as "Information Elements"
13

IPFIX normative documents
RFC 5101: Specification of the IPFIX
Protocol for the Exchange of IP
Traffic Flow Information, 2008
RFC 5102: Information Model for
IPFIX, 2008
RFC 5103: Bidirectional Flow Export
Using IPFIX, 2008
RFC 5473: Reducing Redundancy in
IPFIX and PSAMP Reports, 2009
RFC 5610: Exporting Type
Information for IPFIX
Information Elements, 2009
RFC 5655: Specification of the IPFIX
File Format, 2009
RFC 5815: Definitions of Managed
Objects for IPFIX, 2010 core protocol specification
14

IPFIX informational documents
RFC 3917: Requirements for IPFIX, 2004
RFC 3955: Evaluation of Candidate Protocols for
IPFIX, 2004
RFC 5153: IPFIX Implementation Guidelines, 2008
RFC 5470: Architecture for IPFIX, 2009
RFC 5471: Guidelines for IPFIX Testing, 2009
RFC 5472: IPFIX Applicability, 2009
RFC 5982: IPFIX Mediation: Problem Statement,
2010
15

Current issues in the IPFIX WG
Configuration interface for configuring IPFIX devices defined as YANG module
Mediation particularly for large networks driven by NTT aggregation anonymization
Flow selection
Structuring flow records extending IPFIX capabilities
Using IPFIX for reporting other information
MIB variables, SIP server logs, etc.
16

PSAMP
Established in Summer 2002
Focus on sampling and capturing packets and on transferring them to data collectors
Target applications traffic profiling monitoring network behavior
Extends IPFIX export
Defines packet sampling with much more detail packet filtering and sampling information model
17

IPPM
"The IPPM WG will produce documents that define specific metrics and procedures for accurately measuring and documenting these metrics:" connectivity one-way delay and loss round-trip delay and loss delay variation loss patterns packet reordering bulk transport capacity
(BTC = data_sent / elapsed_time) link bandwidth capacity
Refer to WG official page for list of already published RFCs and ID http://datatracker.ietf.org/wg/ippm/
18

Final remarks
The IETF developed IPFIX as standard protocol for reporting IP flow information
Technology is mature many implementations several interoperability testing events major router vendors expected to release
IPFIX soon as part of standard installation
IPFIX is extensible
BGP-related flow info can already be reported additional information elements can be added
IPFIX can be used to report measurements at peering points appropriate metering hardware required
19