ASTUTE: Detecting a Different
Class of Traffic Anomalies
Fernando Silveira1,2, Christophe Diot1, Nina Taft3,
Ramesh Govindan4
1
Technicolor
2 UPMC Paris Universitas
3 Intel Labs Berkeley
4 University of Southern California
ACM SIGCOMM 2010
A Short-Timescale Uncorrelated-Traffic Equilibrium
ASTUTE: Detecting a Different
Class of Traffic Anomalies
Comparing to Kalman Filter and Wavelet Analysis,
ASTUTE can find anomalies with different features
• Kalman & Wavelet can detect:
 few large flows
• ASTUTE can detect:
 many small flows
Outline

Motivation & Goal

ASTUTE – An Equilibrium Model

ASTUTE-based Anomaly Detection

Experimental Methodology

Performance Evaluation

Conclusion & My Comments
2010/11/2
Speaker: Li-Ming Chen
3
Anomaly Detection

Traffic anomalies (in large ISPs & enterprise networks) come
from:




Malicious activities (e.g., DoS, port scan)
Misconfigurations/failures of network components (e.g., link
failure, routing problem)
Legitimate events (e.g., large file transfers, flash crowds)
Anomaly detection:


2010/11/2
Build a statistical model of normal traffic
An anomaly is defined as deviation from the normal model
Speaker: Li-Ming Chen
4
Motivation: Challenges in Anomaly
Detection

Anomaly Detection:

Pros:


Cons:





Can detect new anomalies!
Training takes times
Training data is never guaranteed to be clean
Periodical (re)training is required
False alarm
 Can we detect anomalies without having to learn what
is normal?
2010/11/2
Speaker: Li-Ming Chen
5
Observation

Network Traffic show Equilibrium:



When many flows are multiplexed on a non-saturated link,
their volume changes over short timescales tend to cancel
each other out
 making the average change across flows close to ZERO
The equilibrium property


Holds if the flows are independent
While, is violated by traffic changes caused by several,
potentially small, correlated flows

2010/11/2
~ traffic anomalies
Speaker: Li-Ming Chen
6
Goal

Propose a new approach to anomaly detection based on
ASTUTE


Advantages:


A mathematical model to describe “A Short-Timescale
Uncorrelated-Traffic Equilibrium”
No training – computationally simple and immune to datapoisoning

Accurately detects a well-defined class of traffic anomalies

Theoretical guarantees on the false positive rates
Evaluate the performance against Kalman filter and
wavelet analysis
2010/11/2
Speaker: Li-Ming Chen
7
Outline

Motivation & Goal

ASTUTE – An Equilibrium Model

ASTUTE-based Anomaly Detection

Experimental Methodology

Performance Evaluation

Conclusion & My Comments
2010/11/2
Speaker: Li-Ming Chen
8
Equilibrium Model
bin i
Measure flow
volume on a link
for each time bin
bin i+1
…
time
flow f starts
at time bin sf
…
xf,i xf,i+1
flow f ’s volume of each time bin
can be represented as a vector:

x f  ( x f ,s f , x f ,s f 1 ,..., x f ,s f d f 1 )
flow f continued
for df bins



Flow: a set of packets that share the same values for a given set of
traffic features (e.g., 5-tuple)
Binning: use time bin to study the evolution of a flow
Flow volume: number of packets in the flow during the
corresponding bin
2010/11/2
Speaker: Li-Ming Chen
9
Equilibrium Model:
Focus on Volume Changes of Flows
bin i
bin i+1
…
time
…
F: set of
flows that
are active
in i or i+1
xf,i xf,i+1
flow f ’s volume of each time bin
can be represented as a vector:

x f  ( x f ,s f , x f ,s f 1 ,..., x f ,s f d f 1 )
 f ,i  x f ,i1  x f ,i (volume change of f from i to i+1)
2010/11/2
Speaker: Li-Ming Chen
10
Consequences of the ASTUTE Model

Assumptions:



Intuition: independent flows
cancel each other out
(A1) Flow independence
(A2) Stationary
Theorem 1 (consequences of the ASTUTE):
other
2010/11/2
Speaker: Li-Ming Chen
11
Outline

Motivation & Goal

ASTUTE – An Equilibrium Model

ASTUTE-based Anomaly Detection

Experimental Methodology

Performance Evaluation

Conclusion & My Comments
2010/11/2
Speaker: Li-Ming Chen
12
ASTUTE-based Anomaly Detection
Method
A toy example :






Set of active flows, F
Mean volume change, ˆi
2
Variance of volume changes, ˆ i
Compute AAV


A detection threshold K(p)
A pair of consecutive time bins
i
Measure:


K ( p)  2
Given:
(ASTUTE Assessment Value):
ˆi
K '
F
ˆ i
i 
0
+2
-1
i+1
ˆi  1 / 3
ˆ i 2  7 / 3
 K '  0.378 K ( p)
Flag an alarm if:

K '  K ( p)
(copy from author’s slides)
2010/11/2
Speaker: Li-Ming Chen
13
Note: About Volume Changes

Requirement:


Only consider traffic on non-saturated links, and using
short-timescale bins
Volume change (for F flows that are active at bin i):

Mean:

Standard deviation:
2010/11/2
Speaker: Li-Ming Chen
14
Note: About Detection Threshold

For large F, ˆi has a (1-p) confidence interval given by the
central limit theorem
ˆ
i
 K ( p) 
<0
>0
ˆ
 ˆ
i
ˆ i
F
 K ( p) 
ˆ i
i
1-p conf. interval
p/2
F
-K(p)



0
K(p)
 If I ˆ contains zero, then F satisfies ASTUTE
i
 Otherwise, there is an ASTUTE anomaly at time bin i
 smallest value of K(p) is
2010/11/2
ˆi
F
ˆ i
(defined as AAV)
Speaker: Li-Ming Chen
15
Note: Situations that ASTUTE is Violated

There are 2 possibilities that ASTUTE is violated:

(1) false positive



Controlled by false positive rate p
In a fraction p of the time bins, ASTUTE may be violated by
normal traffic
(2) Flows violate the model’s assumption: independence &
stationary

Stationary:



Independence:

2010/11/2
Only over the timescale of a typical flow duration
Authors study which bin sizes show stationary behavior
Many flows increase/decrease their volumes at the same time!
Speaker: Li-Ming Chen
16
Note: Validate Stationary Assumption (A2)

Stationary:


In the trace:



Depends on timescale (bin size)
Long scales: daily usage bias
Small scales: no bias!
 We use short
timescales to factor
out violations of
stationarity
2010/11/2
Speaker: Li-Ming Chen
17
Note: Validate “Gaussianity” of AAVs
Study the impact of packet sampling rate
Check
distribution
similarity
2010/11/2
Speaker: Li-Ming Chen
18
Outline




Motivation & Goal
ASTUTE – An Equilibrium Model
ASTUTE-based Anomaly Detection
Experimental Methodology





Competitors (or collaborator!?): Kalman & Wavelet
Inspect anomalies from traffic data and identify their root
causes
Simulation through anomaly injection
Performance Evaluation
Conclusion & My Comments
2010/11/2
Speaker: Li-Ming Chen
19
Kalman & Wavelet (alternative anomaly
detectors for comparison purpose)

Kalman: a spatio-temporal detector




Learning spatial and temporal correlations to predict the
next values
Its threshold parameter has similar semantics to that of
ASTUTE (allowing a direct comparison)
[26] A. Soule, K. Salamatian, and N. Taft, “Combining Filtering and Statistical
Methods for Anomaly Detection,” in Proc. IMC, 2005.
Wavelet: a frequency-based detector



2010/11/2
Decompose signals into low/medium/high frequency bands
The variance of the combined signal (medium & high freq.
bands) represents anomalies
[2] P. Barford, J. Kline, D. Plonka, and A.Ron, “A Signal Analysis of Network Traffic
Anomalies,” In Proc. IMW, 2002.
Speaker: Li-Ming Chen
20
Kalman & Wavelet (cont’d)

Targets of these two detectors:





2010/11/2
(1) packet volume time series
(2) entropy time series of Src. IP
(3) entropy time series of Dst. IP
(4) entropy time series of Src. Port
(5) entropy time series of Dst. port
Speaker: Li-Ming Chen
21
Dataset

Flow traces from 3 different networks
Flow
sampling:
0.1 (between research institutions)
0.01 (public Internet  European NRENs)
NO (inside the enterprise network)
2010/11/2
Speaker: Li-Ming Chen
22
Manual Classification of Anomalies for
Root Cause Analysis

Goal:



Approach:


To perform “root cause” analysis for the anomalies found by
ASTUTE, Kalman, and Wavelet
 need to know the root cause first
Use information provided by ASTUTE to help the process
of manual classification of anomalies in the traffic trace
Steps:



2010/11/2
(1) correlated anomalous flows
(2) anomalous flow identification
(3) anomalous flow classification (by hand)
Speaker: Li-Ming Chen
23
Results of Anomalous Flow Classification

Take these as the criteria for labeling the anomalies
found in the three traces
2010/11/2
Speaker: Li-Ming Chen
24
Simulation through Anomaly Injection

Benefit:



Simulation helps understand how methods trade-off
detection rates for false positives (ROC curves)
ps: for comparing Kalman and ASTUTE only
Approach:


2010/11/2
For end-host activity: build a set of benchmark anomalies
and inject (recreate identified anomalies)
For outages: remove related traffic
Speaker: Li-Ming Chen
25
Outline

Motivation & Goal

ASTUTE – An Equilibrium Model

ASTUTE-based Anomaly Detection

Experimental Methodology

Performance Evaluation

Conclusion & My Comments
2010/11/2
Speaker: Li-Ming Chen
26
Number of Anomalies and Anomaly
Overlap
Small overlap
Kalman & Wavelet
have more overlap
among each other
• what are these anomalies??
2010/11/2
Speaker: Li-Ming Chen
27
Anomaly Types (Internet2)
Detection capabilities
are different
2010/11/2
Speaker: Li-Ming Chen
28
Anomaly Types (GEANT2 & Corporate)
Users characteristics
in different networks
are different
2010/11/2
Speaker: Li-Ming Chen
29
Small Detector Overlap
Kalman/Wavelet
(few large flow)
Less total volume
ASTUTE
(several small flow)
(map qualitative properties (types) of the anomalies
2010/11/2
to
their quantitative properties Speaker:
(# flows
Li-Mingand
Chen packets))
30
Detection Performance
2010/11/2
Type 1
Type 2
Type 2
Type 3
Speaker: Li-Ming Chen
31
Complementarity of ASTUTE & Kalman
After combination, the performance is better!
2010/11/2
Speaker: Li-Ming Chen
32
Outline

Motivation & Goal

ASTUTE – An Equilibrium Model

ASTUTE-based Anomaly Detection

Experimental Methodology

Performance Evaluation

Conclusion & My Comments
2010/11/2
Speaker: Li-Ming Chen
33
Conclusion

ASTUTE detects anomalies w/o learning the normal
behavior



Computationally simple and immune to data-poisoning
Specializes on strongly correlated flows (several small flow)
Limitation: can not find anomalies involving a few large
flows



2010/11/2
But those are easy to find!
ASTUTE and Kalman complement each other nicely
ASTUTE also provides information that is useful to perform
root cause analysis
Speaker: Li-Ming Chen
34