Defending Distributed Systems
Against Malicious Intrusions and
Network Anomalies
Kai Hwang
Internet and Grid Computing Laboratory
University of Southern California
Keynote Presentation
at the IEEE International Workshop on Security in Systems and Networks (SSN-2005),
held in conjunction with the IEEE International Parallel and Distributed Processing
Symposium (IPDPS-2005), Denver, Colorado, April 8, 2005
This presentation is based on research findings by USC GridSec team. Project Web site:
http://GridSec.usc.edu, supported by NSF ITR Grant No. 0325409, and contributed by
Min Cai, Shanshan Song, Ricky Kwok, Ying Chen, and Hua Liu
1
Presentation Outline:




Security/privacy demands in networked
or distributed computer systems
GridSec NetShield architecture for defending
distributed resource sites in Grids, clusters, etc.
Internet datamining for collaborative anomaly and
intrusion detection system (CAIDS) with traffic
episode rule training and analysis
Fast containment of internet worm outbreaks and
tracking of related DDoS attacks with distributedhashing overlays
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
2
Security and Privacy Demands in
Network and Distributed Systems






Trusted resource allocation, sharing, and scheduling
Secure communications among resource sites, clusters,
and protected download among peer machines
Intrusion and anomaly detection, attack repelling, trace
back, pushback of attacks, etc
Fortification of hardware/software (firewalls, packet filters,
VPN gateways, traffic monitors, security overlays, etc. )
Self-defense toolkits/middleware for distributed defense,
risk assessment, worm containment, response automation
Anonymity, confidentiality, data integrity, fine- grain access
control, resolving conflicts in security policies, etc
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
3
GridSec: A Grid Security ITR Project at USC
Site S1
Host
3
VPN
Gateway
3
Host
Internet
3
Host
2
3
3
Host
2
3
Host
3
1
Site S2
Host
3
VPN
Gateway
Host
Host
3
VPN
Gateway
3
Host
Steps for automated self-defense at resource site :
Step 1: Intrusion detected by host-based firewall /IDS
Step 2: All VPN gateways are alerted with the intrusions
Step 3: Gateways broadcast response commands to all hosts
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
4
Site S3
The NetShield Architecture with Distributed
Security Enforcement over a DHT Overlay
Invoke Response
Broadcast
Update Prevalence
Flood control for
DDoS Defense
Worm Signature
Generation
Signature Update
Collaborative
Alert Correlation
WormShield and DDoS defense
Misuse
Detection
Anomaly
Detection
CAIDS
Distributed Intrusion Detection/Response System
Intrusion Detection
Information Exchange
DHT-based
Overlay Network
Security Policy
Implementation
Overlay Network for Trust Management
Authentication
Authorization
Delegation
Integrity Control
Trust Integration/Negotiation Platform Overlay
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
5
Building Encrypted Tunnels between Grid
Resource Sites Through the DHT Overlay

The number of encrypted tunnels should grow with O(N)
instead of O(N x N), where N is the number of Grid sites

Using shortest path, security policy is enforced
with minimal VPN tunnels to satisfy special Grid
requirements, automatically

How to integrate security policies from various private
networks through the public network ?

How to resolve security policy conflicts among hosts,
firewalls, switches, routers, and servers, etc. in a Grid
environment ?
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
6
Trust Integration over a DHT Overlay
V
Site S3
V
Site S2
Site S1
Physical backbone
Site S4
DHT Overlay Ring
V
V
V
VPN
Gateway
SeGO
Server
Trust Vector
Trust vector
propagation
User application and
SeGO server
negotiation
Hosts
Cooperating gateways working together
to establish VPN tunnels for trust integration
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
7
USC NetShield Intrusion Defense System
for Protecting Local Network of
Grid Computing Resources
ISP
The
Internet
April 8, 2005, Kai Hwang
Network
Router
The
NetShield
System
Firewall
Datamining for Anomaly
Intrusion Detection (IDS)
Risk
Assessment
System (RAS)
http://GridSec.usc.edu
Intrusion
Response
System (IRS)
8
Victim’s
Internal
Network
Alert Operations performed in local Grid
sites and correlated globally
Local alert correlation
Global alert correlation
DHT module
Global alert clustering
Alert classification
Alert merging
Alert formatting
Alert correlation
Alert clusters
Local alert clustering
Alerts
IDS
IDS
IDS
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
Intrusion reports
Alert Assessment
Reporting, and Reaction
9
Basic Concept of Internet Episodes

Event Type: A, B, C, D, E, F, etc.

Event Sequence: e.g., <(E,31),(D,32),(F,33)>

Window: Event sequence with a particular width

Episode: partially ordered set of events, e.g. whenever A occurs, B
will occur soon

Frequency of episode: fraction of windows in which episode occurs

Frequent episode: set of episodes having a frequency over a
particular frequency threshold

Frequent episode rules are generated to describe the
connection events
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
10
Frequent Episode Rules (FER)
for Characterizing Network Traffic Connections
E → D, F ( c, s )
The episode of 3 connection events (E, D, F) = (http, smtp, telnet).
On the LHS , we have the earlier event E (http). On the RHS, we have
two consequence events D (smtp) and F(telnet); where s is the
support probablity and c is the confidence level specified below:
(service = http, flag = SF)
→
(service = smtp, srcbyte = 5000),
(service = telnet, flag = SF) (0.8, 0.9)
Support probability s = 0.9 and Confidence level c = 0.8 that the
episode will take place in a typical traffic stream
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
11
A Cooperative Anomaly and Intrusion Detection System
(CAIDS), built with a Network Intrusion Detection System
(NIDS) and an Anomaly Detection System (ADS) operating
interactively through automated signature generation
Training data from
audit normal traffic
records
Single-connection attacks
detected at packet level
Audit records
from traffic data
IDS
Known
attack
signatures
from ISD
provider
April 8, 2005, Kai Hwang
Signature
Matching
Engine
Attack
Signature
Database
Unknown
or burst
attacks
New
signatures
from
anomalies
detected
Episode Rule
Database
Episode
Mining
Engine
Anomalies detected
over multiple
connections
ADS
Signature
ADS
http://GridSec.usc.edu
Generator
12
Internet Datamining
for Episode Rule Generation
Audit data
Feature
extraction
Connection
Records
Training
phase
Attack-free
episode rules
Episode rule
mining Engine
Detection
phase
April 8, 2005, Kai Hwang
Normal
profile
database
Rules from
real-time
traffic
Anomaly
Detection
Engine
http://GridSec.usc.edu
Alarm
Generation
13
Attack Spectrum from MIT Lincoln
Lab in 10 Days of Experimentation
Attack num bers
20
15
DoS
U2R
10
R2L
5
Probe
0
Day1
Day2
Day3
Day4
Day5
Day6
Day7
Day8
Day9
Days
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
14
Day10
Automated Signature Generation from
Frequent Episode Analysis
1. Label relevant connections to
associate with an FER.
Online traffic episode rules
from the datamining engine
Episode rules
matching the normal
FER database ?
Yes
Episode Frequency
exceeding the rule
threshold ?
No
Yes
No
(Massive attacks)
2 Calculate additional information such
as connection count, average and
percentage of connections, etc.
3 Select one of the predefined classifiers
4 Use the selected classifier to classify the attack class
and find the relevant connections
5 Extract common features in all identified
connections, such as the IP addresses, protocol, etc. to
form the signature
(Stealthy attacks)
2 Check error flags or other useful
temporal statistics
3 Extract common features such
as IP addresses, protocol, etc.
to form the signature
Adding new
signatures to the
Snort database
Ignore the normal episode rules from legitimate users (No anomaly detected)
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
15
Successful Detection Rates of Snort , Anomaly Detection
System (ADS), and the Collaborative Anomaly and
Intrusion Detection System (CAIDS)
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
16
False Alarms out of 201 Attacks in CAIDS
Triggered by Different Attack Types
under Various Scanning Window Sizes
Number of False Alarms
18
16
14
R2L
DoS
Pr obe
U2R
12
10
8
6
4
2
0
100
300
500
1000
7200
Window Size (Second)
Using larger windows result in more false alarms. Shorter windows in 300 sec
or less are better in the sense that shorter episodes will be mined to produce
shorter rules, leading to faster rule matching in the anomaly detection process
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
17
Intrusion Detection Rate
(%)
Detection Rates of Snort, ADS, and CAIDS
under Various Attack Classes
70
60
50
40
SNORT
ADS
CAI DS
30
20
10
0
DoS
U2R
R2L
PROBE
Tot al
At t ack Types
On the average, the CAIDS (white bars) outperforms
the Snort and ADS by 51% and 40%, respectively
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
18
ROC Curves for 4 Attack Classes
on The Simulated CAIDS
Intrusion Detection Rate (%)
80
70
60
DoS
Pr obe
R2L
U2R
50
40
30
20
10
0
0
2
4
6
8
10
12
False Alarm Rate (%)
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
19
Intrusion Detection Rate (%)
ROC Performance of Three
Intrusion Detection Systems
80
70
60
50
40
30
CAI DS
Snor t
ADS
20
10
0
0
2
4
6
8
12
10
False Alarm Rate (%)
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
20
Internet Worm and Flood Control:



A DHT-based WormShield overlay network is under
development at USC.
Fast worm signature generation and fast
dissemination through both local and global
address dispersion
Automated tracking of DDoS attack-transit routers
to cut off malicious packet flows for dynamic DDoS
flood control
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
21
The WormShield Built with a DHT-based
Overlay with Six Worm Monitors
Local Table:
Chord ID
76
112
55
215
Global Table: Tp  10,Tc  20
Chord ID
Identified
Worm
Signature!
215
180
...
Content
Block
s4
s5
Global
Address
Prevelance Dispersion
5+6=11
18
4+8=12
22
Tl  3
Content
Block
s1
s2
s3
s4
0/256
Local
Prevelance
1
4
2
5
(src, dest)
Addresses
S1(A), D1(A)
S2(A), D2(A)
S3(A), D3(A)
S4(A),D4(A)
Site A
Site B
Site E
Local Table:
Chord ID
180
nce }
vela (A)
Pre A), D4
a te
(
Upd , 5, S4
4
5, s
{21
192
U
{21 pdate
5, s
P
4, 5 revela
, S4
n
(A ), c e
D4(
A )}
Site F
Site C
Tl  3
Local Table:
Content
Local
(src, dest)
Block Prevelance Addresses
S5(D),D5(D)
s5
7
Site D
April 8, 2005, Kai Hwang
64
Tl  3
Content
Local
Chord ID
Block Prevelance
215
s4
6
180
s5
4
128
http://GridSec.usc.edu
22
(src, dest)
Addresses
S4(C),D4(C)
S5(C),D5(C)
The WormShield Signature Generation Process
Monitored DMZ Traffic
Rabin Fingerprinting
G
lo A
bd
ad
lCr
e
Chord
ID
s
Update P(j), C(j) o
ns
te DID(j)
n is
t p
Pe
re rs
v
No
a io
le n
nT
aYes
c
eb
P(j) > Tp
& le
&& C(j) > T
Content Block j
L
o
c
a
Local
lC Chord ID Content
Block
Prevelance
No
o
n
ID(j)
j
L(i, j)
L(i, j)> Tl
te
n
Update L(i,j)
t
P
re
Yes
v
a
leL
c
no
c
c
ea
Content
l
SRC IP
DEST IP
T
Block
No
aA
d
bd
j
S(i,
j)
D(i,
j)
|S(i,j)|+|D(i,j)|> Ts
ler
Update
e
s
S(i,j), D(i, j)
s
D
Yes
is
p
e
rs
io
n
T
Send updates
Process updates
a
b
for P(j) and C(j) to
for P(j) and C(j) from
le
Other
monitor
root(j)
other monitors
WormShield
Monitors
http://GridSec.usc.edu
Address
Dispersion
C(j)
Report j as
suspected worm
Disseminate suspected
worm signature j to
WormShield network
Chord Protocol
April 8, 2005, Kai Hwang
Global
Prevelance
P( j)
23
Signature Detection in Worm Spreading and the
Growth of Infected hosts for Simulated CodeRed
Worms on a Internet Configuration of 105,246 Edge
networks in 11,342 Autonomous Systems
Containing 338,652 Vulnerable Hosts
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
24
Effects of Local Prevalence Threshold
Worm spreading and the growth of infected hosts
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
25
Effects of Global Address Prevalence
on
Worm Spreading and the Growth of Infected Hosts
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
26
Number of infected hosts at detection time
Reduction of Infected Hosts by
Independent vs. Collaborative
Monitoring over the Edge Networks
Average of independent monitors
Best of independent monitors
Collaborative monitors in WormShield
300000
250000
200000
150000
100000
50000
0
61(0.1%)
612(1%)
6121(10%)
30608(50%)
Number of edge networks monitored
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
27
LogLog
Cardinality
Summary
Ingress
Router
Identified
as an ATR
Tracking and
Flood Control
Attack Flows
LogLog
Cardinality
Summary
Legitimate Flow
Attack Flows
Legitimate Flow
Packet/Flow Counting for Tracking
Attack-Transit Routers (ATRs)
Ingress
Router
Legitimate Flows
LogLog
Cardinality
Summary
Ingress
Router
Identified
as an ATR
LogLog
Cardinality
Summary
Tracking and
Flood Control
LogLog
Cardinality
Summary
LogLog
Cardinality
Summary
Packet-level Traffic Matrix A
Last Hop Router
Flow-level Traffic Matrix B
Victim
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
28
False Positive Rate of Identified ATRs
The false positive of
attack-transit routers(%)
100
70 % Percentile
80 % Percentile
90 % Percentile
80
60
40
20
0
0.0
0.2
0.4
0.6
0.8
1.0
The ratio of legitimate trafic to attack traffic
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
29
Other Hot Security Research Areas:

Efficient and enforceable trust models are very much in
demand for networked and distributed systems: PKI
services, VPN tunneling, trust negotiation, security overlays,
reputation system etc.

Large-scale security benchmark experiments in open Internet
environments are infeasible. The NSF/HSD DETER testbed
should be fully used in performing such experiments to
establish sustainable cybertrust over all edge networks.

Internet datamining for security control and for the guarantee
of Quality-of-Service in real-life network applications –
Interoperability between wired and wireless networks is
a wide-open area for further research.
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
30
Final Remarks



The NetShield built with DHT-based security overlay
networks support distributed intrusion and anomaly
detection, alert correlation, collaborative worm
containment, and flooding attack suppression.
The CAIDS can cope with both known and unknown
network attacks, secure many cluster/Grid/P2P
operations in using common Internet services: telnet,
http, ftp, Email, SMTP, authentication, etc.
Automated virus or worm signature generation plays a
vital role to monitory network epidemic outbreaks and to
give early warning of large-scale system intrusions,
network anomalies, and DDoS flood attacks. Extensive
benchmark experiments on the DETER test bed will
prove the effectiveness.
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
31
Recent Related Papers:
1.
M. Cai, K. Hwang, Y. K. Kwok, Y. Chen, and S. S. Song, “Fast
Containment of Internet Worms and Tracking of DDoS Attacks with
Distributed-Hashing Overlays”, IEEE Security and Privacy, accepted
to appear Nov/Dec. 2005.
2.
K. Hwang, Y. Kwok, S. Song, M. Cai, R. Zhou, Yu. Chen, Ying. Chen,
and X. Lou, “GridSec: Trusted Grid Computing with Security Binding
and Self-Defense against Network Worms and DDoS Attacks”,
International Workshop on Grid Computing Security and Resource
Management (GSRM’05), in conjunction with ICCS 2005, Atlanta, May
22-25, 2005.
3.
M. Qin and K. Hwang, “Frequent Episode Rules for Internet Traffic
Analysis and Anomaly Detection”, IEEE Network Computing and
Application Symp. (NCA-2004), Cambridge, MA. August 31, 2004
4.
K. Hwang, Y. Chen and H. Liu, “ Defending Distributed Computing
Systems from Malicious Intrusions and Network Anomalies”, IEEE
Workshop on Security in Systems and Networks (SSN’05), in
conjunction with IEEE IPDPS 2005, Denver, April 8, 2005.
April 8, 2005, Kai Hwang
http://GridSec.usc.edu
32