Network Security
Part III: Security Appliances
Intrusion
Detection Systems
Why this talk?
• IDS solutions are not perfect
• IDS administrators are not perfect
• Security is a process!
– Not a person!
– Not a product!
– Intrusion detection is a part of information security !!!
2
SECURITY INNOVATION ©2003
The Problem
• Present network speeds and topology have
made it difficult and expensive to deploy a
pervasive IDS.
3
SECURITY INNOVATION ©2003
Where does IDS fit?
• IDS are useful as an additional layer of
defense, no more
• IDS are helpful when advanced attackers are
attacking you with new attacks
• Two major types today: network IDS (snort)
and host IDS (AIDE, log watcher, etc)
• Missing IDS type: application IDS
– eEye’s SecureIIS might be a precursor, but has
been proven flawed already
– AZN-API is a useful new direction for
authorization issues
4
SECURITY INNOVATION ©2003
Generic Issues with IDS
• It’s yet another system that has to be
monitored
• Yet another set of logs that will be ignored
– Too verbose?
– Not sensitive enough?
– Not enough eyes to monitor all your systems?
• The “three cries and you’re out” problem
– No one likes being woken up continuously at 3
am
5
SECURITY INNOVATION ©2003
Types of IDS’
•
•
•
•
•
6
Plain Hard Work
Host Based
Network Based
Log Based
Target Monitoring
SECURITY INNOVATION ©2003
Plain Hard Work
•
•
•
•
•
•
7
Freeware
Sniffers
Log analysis
Lots of time
Very exciting work
Log aggregation is a pain
SECURITY INNOVATION ©2003
Log Based
•
•
•
•
8
Reviews syslog
Reviews SNMP
Not Real-time
Forensics Tool
SECURITY INNOVATION ©2003
Target Monitoring
•
•
•
•
•
9
Watches the OS
Lives on Box
Watches Files
Scheduled Runs
Near Real-time
SECURITY INNOVATION ©2003
Network Based
• Listens to All Traffic on Segment
• Must Live on Target Net
• Has Throughput Limitations especially in a
100Mb/s traffic environment
10
SECURITY INNOVATION ©2003
Network IDS
• Usually has one or more interfaces in promiscuous
mode – which makes them detectable in certain
circumstances (see anti-sniff)
• Useful to spot unusual traffic trends
• Even with the fastest processors, most commercial
and non-commercial network IDS cannot cope with
> 100 Mb/s traffic
• Good example: snort
• Issue: useful only if you can monitor it and the
alarms have been calibrated to suit your needs
11
SECURITY INNOVATION ©2003
Network IDS
•
•
•
•
Searches for patterns in packets
Searches for patterns of packets
Searches for packets that shouldn't be there
May ‘understand’ a protocol for effective pattern
searching and anomaly detection
• May passively log, alert with SMTP/SNMP or
have real-time GUI
12
SECURITY INNOVATION ©2003
Network IDS Limitations
•
•
•
•
•
•
•
13
Obtaining packets - topology & encryption
Number of signatures
Quality of signatures
Performance
Network session integrity
Understanding the observed protocol
Disk storage
SECURITY INNOVATION ©2003
Network IDS
/cgi-bin/phf
14
Jane used the PHF
attack!
SECURITY INNOVATION ©2003
Network IDS
NMAP
15
Jane did a port
sweep!
SECURITY INNOVATION ©2003
Host Based
•
•
•
•
•
•
16
Lives on Host
Uses CPU Cycles
Uses Disk Cycles
Real-time Alerts
Many Vendors
Thresholds
SECURITY INNOVATION ©2003
Host IDS
• Host based IDS perform a range of useful integrity
tests, such as tracking file system changes
• WinNT/2K: prefer auditing to tripwire (or maybe
use both) – auditing is real time, and you know
which user caused the event as they are doing it
• Tripwire and AIDE are non-real time and only let
you know something has happened after the fact
• Commercial host IDS do way more than open source
IDS today, but expect this to change soon
17
SECURITY INNOVATION ©2003
Host Based IDS
• Signature log analysis
– application and system
• File integrity checking
– MD5 checksums
• Enhanced Kernel Security
– API access control
– Stack security
• Network Monitoring Hybrids
18
SECURITY INNOVATION ©2003
Host Based IDS Limitations
• Places load on system
• Disabling system logging
• Kernel modifications to avoid file integrity
checking (and other stuff)
• Management overhead
• Network IDS Limitations
19
SECURITY INNOVATION ©2003
Host Based IDS
messages
xfer
access_log
secure
sendmail
20
SECURITY INNOVATION ©2003
Host Based IDS
messages
xfer
One
Security
Log
access_log
secure
sendmail
21
SECURITY INNOVATION ©2003
Application IDS
• Doesn’t exist … but should!
• Requires the assistance of applications to really
function correctly
• There isn’t a general purpose API to implement this,
and many product writers believe that they are
writing secure software, so…
22
SECURITY INNOVATION ©2003
Where to deploy IDS
• The typical place is in the DMZ or behind the
firewall
• There’s too many lame attacks for IDS to be
out in no man’s land
• Much more useful to see those attacks that
have penetrated your firewall or are in a
sensitive network
23
SECURITY INNOVATION ©2003
Firewalls as an IDS
• Excellent source of network probe, attack and
misuse information
• Detect policy deviations based on access
control lists
• Some have “NIDS” capabilities
24
SECURITY INNOVATION ©2003
Network Honeypots
• Sacrificial system(s) or sophisticated
simulations
• Any traffic to the honeypot is considered
suspicious
• If a scanner bypassed the NIDS, HIDS and
firewalls, they still may not know that a
Honeypot has been deployed
25
SECURITY INNOVATION ©2003
Internet
Router/Firewall
Bastion
Bastion Hosts
Hosts
Mail
FTP
Router/Firewall
Internal Network
26
SECURITY INNOVATION ©2003
Technical Bypass Techniques
• NIDS
–
–
–
–
–
–
• HIDS
fragmentation
TCP un-sync
Low TTL
‘Max’ MTU
HTTP Protocol
Telnet Protocol
– Kernel Hacks
– Bypassing stack
protection
– Library Hacks
– HTTP Logging
Packet Insertion Techniques
27
SECURITY INNOVATION ©2003
NIDS
IP #1
Session #1
IP #2
Session #2
IP #3
Session #3
SESSION QUEUE
FRAGMENT QUEUE
NIDS
28
SECURITY INNOVATION ©2003
NIDS
IP #1
Session #1
IP #2
Session #2
IP #3
Session #3
SESSION QUEUE
FRAGMENT QUEUE
NIDS
29
SECURITY INNOVATION ©2003
Bypassing NIDS - Fragmentation
• NIDS must reconstruct fragments
– Maintain state = drain on resources
– Must overwrite correctly = more drain on resources
•
•
•
•
30
Target server correctly de-frags
Attack #1 - just fragment
Attack #2 - frag with overwrite
Attack #3 - start an attack, follow with many
false attacks, finish the first attack
SECURITY INNOVATION ©2003
Bypassing NIDS - TCP un-sync
• Inject a packet with a bad TCP checksum
– fake ‘FIN’ packet
• Inject a packet with a weird TCP sequence
number
– step up
– wrapping numbers
31
SECURITY INNOVATION ©2003
Bypassing NIDS - Low TTL
WWW
NIDS
3
32
2
SECURITY INNOVATION ©2003
1
Bypassing NIDS - Max ‘MTU’
WWW
NIDS
Segment with
MTU = 1300
1350 byte
packet with
DF = 1
33
SECURITY INNOVATION ©2003
Bypassing NIDS - HTTP Proto
•
•
•
•
•
•
•
34
‘/’ padding: “/cgi-bin///phf”
Self referencing directories: “/cgibin/./phf”
URL Encoding: “%2fcgi-bin/phf”
Reverse Traversal: “/cgi-bin/here/../phf”
TAB instead of spaces removal
DOS/Win syntax: “/cgi-bin\phf”
Null method: “GET%00/cgi-bin/phf”
SECURITY INNOVATION ©2003
Bypassing NIDS - Telnet Proto
• Strip out Telnet codes
• Automatic proxies which add random
characters followed by backspace
– “su X{backspace}root”
35
SECURITY INNOVATION ©2003
Bypassing NIDS - Resources
• Tools
– Whisker - Rain Forest Puppy
http://www.wiretrip.net/rfp/p/doc.asp?id=21&iface=2
– Fragrouter - Dug Song
http://www.anzen.com/research/nidsbench/
– Congestant - horizon, Phrack 54
• Papers
– “Insertion, Evasion and Denial of Service: Eluding Network
Intrusion Detection”, Tom Ptacek, Timothy Newsham
http://secinf.net/info/ids/idspaper/idspaper.html
– Bro information: ftp://ftp.ee.lbl.gov/papers/bro-CN99.ps.gz
36
SECURITY INNOVATION ©2003
Bypassing HIDS - Kernel
Hacks
• Windows NT
– 4 byte patch that removes all security restrictions
from objects within the NT domain.
– Could use access to disable or manipulate HIDS
• Linux - “itfs.c” - kernel module
- not in /proc/modules
- hides a sniffer
- hides files
- hides processes
37
- redirects execve()
- socket backdoor
- magic setuid gets root
SECURITY INNOVATION ©2003
Bypassing HIDS - Stack
Protection
• Stackguard
–
–
–
–
A ‘canary’ is placed next to return address
Program halts and logs if canary is altered
Canary can be random or terminating
Bypass: overwrite return address without touching
canary
– Fix: XOR the return address and the canary
– Point: Yet another example of an arms race
38
SECURITY INNOVATION ©2003
Bypassing HIDS - Library
Hacks
• Environment variables which redirect shared
library locations
• Library has a ‘wrapper’ run by a privileged
program
• Two choices
– Provide certain APIs with original copies of Trojan
files
– Redirect certain APIs to completely different files
39
SECURITY INNOVATION ©2003
Bypassing HIDS - HTTP
Logging
• The anti-NIDS HTTP techniques also may
work for host based IDS tools which do log
analysis
40
SECURITY INNOVATION ©2003
Bypassing HIDS - Resources
•
•
Phrack 51
– “Shared Library Redirection Techniques”,halflife,<halflife@infonexus.com>
– “Bypassing Integrity Checking Systems”,halflife,<halflife@infonexus.com>
Phrack 52
–
•
Phrack 55
–
•
41
“A real NT Rootkit, patching the NT Kernel”, Greg Hoglund <hoglund@ieway.com>
Phrack 56
–
–
–
•
“Weakening the Linux Kernel”, plaguez <dube0866@eurobretagne.fr>
“Shared Library Call Redirection via ELF PLT Infection”, Silvio Cesare
“Backdooring Binary Objects”, <klog@promisc.org>
“Bypassing Stackguard and Stackshield”, Bulba & Kil3r <lam3rz@hert.org>
Stackguard - http://www.immunix.org/documentation.html
SECURITY INNOVATION ©2003
Practical Bypass Techniques
• NIDS
–
–
–
–
–
42
• HIDS
identifying
avoiding
overwhelming
“slow roll”
“distributed scanning”
– identifying
– log deletion
– log modification
• Generic
– Social
– DOS
SECURITY INNOVATION ©2003
NIDS - Identifying
•
•
•
•
•
43
Is it in DNS?
Does it shoot down connections?
Is the sniffing interface detectable?
Is it running on a big red box labeled “IDS”?
Can the alert messages be observed?
SECURITY INNOVATION ©2003
NIDS - Identifying
• Any open ports that match a known IDS?
• Has the target posted to an IDS saying, “We
use product XYZ?”
• Do they have a “This site protected by XYZ”
message on their web site?
44
SECURITY INNOVATION ©2003
NIDS - Avoiding
• Are there other routes into the network?
– Is there an encrypted path?
– Modem dial in?
– Alternate transport layer? (GRE ???)
• Is there an attack not detected by the IDS?
• Is there a technical bypass technique that is
not detected by the IDS?
45
SECURITY INNOVATION ©2003
NIDS - Overwhelming
• Send as many false attacks as possible while
still doing the real attack
– May overload console
– May drop packets
– Admins may not believe there is a threat
• Send packets that “cost” the NIDS CPU cycles
to process
– Fragmented, overlapping, de-synchronized web
attacks with the occasional bad checksum
46
SECURITY INNOVATION ©2003
NIDS - ‘Slow Roll’
• Port scans and sweeps
–
–
–
–
47
Obvious: incremental destination ports
Trivial: randomized ports
Sweep: one port and many addresses
Stealthy: random ports and addresses over time
SECURITY INNOVATION ©2003
Target Mapping
P
o
r
t
s
Plotting all destination
ports from one source IP
to a target network …
Port scan
IP addresses
48
SECURITY INNOVATION ©2003
Port sweep
Target Mapping
random
Simple port walk
Still maps out a
network with
one IP address
P
o
r
t
s
IP addresses
49
SECURITY INNOVATION ©2003
Target Mapping
SLAVES
SLAVES
MASTER
Target sees traffic
from many addresses
50
SECURITY INNOVATION ©2003
HIDS - Identifying
• Almost always after on a system ...
•
•
•
•
•
51
Is there anything in the system logs?
What ports are open?
What is running out of CRON?
What is in the NT registry?
What programs are running?
SECURITY INNOVATION ©2003
HIDS - Logs
• Simple log deletion may be possible
• Simple log altering may also be possible
– replace IP addresses to mislead
– delete key logs
• Logging may be disabled or intercepted
– Removing syslog from services
52
SECURITY INNOVATION ©2003
Generic - Social
• Physical access
• Obtaining “official” access
• Getting others to hack/scan site for you
– IRC & chat groups
– Hacker challengers
• Run the IDS ……
53
SECURITY INNOVATION ©2003
Generic - DOS
• Find the main ‘server’
• Kill it
– IP Bomb
– Port bomb
– IDS DOS
• Find the clients
54
SECURITY INNOVATION ©2003
Drawbacks
•
•
•
•
•
55
Each System has Drawbacks
Some are not Fast Enough
Some are not Real-time
Some Intrude on OS
Others Can Cause Application Compatibility
Problems
SECURITY INNOVATION ©2003
What’s wrong with security?
• All software has defects
–
–
–
56
Best practice says that software can only hope to have as
few as one defect per 1 KLOC
Normal code has 5-15 bugs per 1000 lines
Windows NT has 17 million lines…. Do the math
SECURITY INNOVATION ©2003
Risk model
Cost of attack vs frequency of attack
f
57
$
SECURITY INNOVATION ©2003
Insurance – Mega Corps
• In large corporations, insurance is a method to assign
the risk of catastrophic events to another entity
• Most large corporations are self insuring for most
risks (for example, one of my clients simply pays for
all car accidents; it’s just cheaper that way)
• Most large corporations do not see the point in
insuring an intangible risk such as a web defacement,
but they might insure good will.
58
SECURITY INNOVATION ©2003
External Threats vs Internal
Threats
• Old thinking: Seasoned attacker with extreme
skills will be attacking me every time
• Reality #1: script kiddies will launch zillions
of RDS attacks at you, even though you
might be running Solaris
• Reality #2: your staff are much more of a risk
than the script kiddies of this world
59
SECURITY INNOVATION ©2003
Anatomy of a Script Kiddie Attack
Collect tools
Tag & Brag
Attack victims
60
SECURITY INNOVATION ©2003
Anatomy of a Gifted Amateur
Attack
61
SECURITY INNOVATION ©2003
Anatomy of a strong attack
62
SECURITY INNOVATION ©2003
Internet Age Threats
• Real threats arise from people with motive
• Most external attacks are simple, but not all
• Most successful attacks are essentially
internal fraud
– Audit controls will help
• It is nearly always easier to socially engineer
from within than attack a system from
without once minimum defenses are added
63
SECURITY INNOVATION ©2003
•
•
•
•
•
•
•
•
•
•
•
•
•
Research Challenges
Detect a wide variety of intrusion types
Very high certainty
Real-time detection
Develop a network-wide view rather than local views
Analysis must work reliably with incomplete data
Detect unanticipated attack methods
Scale to very large heterogeneous systems
What data to collect for maximal effectiveness;
network instrumentation
Automated response
Discover or narrow down the source of an attack
Integrate with network management and fault
diagnosis
Infer intent; forming the big picture
Cooperative problem solving
SECURITY INNOVATION ©2003
Methods Under Investigation
• Methods to detect highly unusual events or
combinations of events
– Statistical methods
– Neural networks
– Machine learning
• Methods to detect activity outside prescribed bounds
– Specification-based detection
• New knowledge-based
analysis techniques
Discrepancy
• Traceback methods
– Thumbprinting
65
SECURITY INNOVATION ©2003
Structural
Statistical
– Graphical intrusion Acceptable
detection
– State transition models
(model-based detection) Illegal
Model/Pattern
Profile
Match
Cooperating Detectors
IDS
IDS
IDS
IDS
IDS
Sensors
Also needed:
Efficient and effective methods for peer-to-peer cooperative problem
solving to be applied to the detection problem
–To filter events of only local concern
–To assess a larger “region”
66
SECURITY INNOVATION ©2003
Advanced Techniques
• Statistical anomaly detection (SRI, CMU)
– establish a historical behavior profile for each desired entity (e.g.,
user, group, device, process)
– compare current behavior with the profiles
– detects departures from established norms
– continuously update profiles to “learn” changes in subject
behavior
– addresses unanticipated intrusion types
• Early statistical studies:
– SRI study (Javitz et al):
• Showed users could be distinguished from each other
based on patterns of use
– Sytek study (Lunt et al):
• Showed behavior characteristics can be found that
discriminate between normal user behavior and
simulated intrusions
67
SECURITY INNOVATION ©2003
Advanced Techniques cont’d
• Machine learning (LANL)
– Builds a massive tree of statistical “rules” (typically 100,000’s of
them)
– Branches are labeled with conditional probabilities
– Prunes the tree to a maximum depth of four to six
– Low-occurrence branches are combined
– Tree is “trained” from a few days of data
– Tree cannot be updated to “learn” as usage patterns change
– Activity is considered abnormal if it does not “match” a branch in
the tree or if it matches a branch with low conditional probability
last node
• Meta-Learning (Columbia University)
– Meta-learning integrates a number of separately learned classifiers
– Multi-layered approach:
• machine learning and decision procedures detect intrusions locally
• meta-learning and decision procedures to integrate the
knowledge acquired by the local agents
68
SECURITY INNOVATION ©2003
collective
Advanced Techniques cont’d
• Computational immunology
– based on biological analogies (e.g., self vs. non-self
discrimination)
– build up a database of observed short sequences of
system calls for a program and detect when the
observed program behavior exhibits short
sequences not in that database (U. of NM)
– allows the detection of tampered or malicious
programs or other suspicious events
– this potentially lightweight method is being
implemented in small, autonomous agents in a
CORBA environment (ORA)
69
SECURITY INNOVATION ©2003
Advanced Techniques cont’d
• Model-based detection
– Detects suspicious state transitions (UC Santa Barbara)
• specifies penetration scenarios as a sequence of actions
• keeps track of interesting “state changes”
• attempts to identify attacks in progress before damage is done
– Adapt model-based diagnosis, which has been successful in
diagnosing faults in microprocessors, to intrusion detection
(MIT)
• Graphical detection (UC Davis)
– detects intrusions whose activity spans many machines that
could be difficult to detect locally
– specifies intrusion scenarios as graphs of actions covering
many machines
– the graphs provide an intuitive visual display
70
SECURITY INNOVATION ©2003
Advanced Techniques cont’d
• Signalling Infrastructure Detection (GTE)
– detect anomalous events in a network and signalling
infrastructure typical of telephone service providers
– designed for integration into network operations centers
– uses existing systems/tools for data collection
– uses anomaly detection and specific signalling protocol
“sanity checks”
• Detection in high-speed networks (MCNC)
– Integrates anomaly detection techniques with network
management for ATM networking (IP over ATM)
– Logical analysis of routing protocol operation to detect
anomalous states
71
SECURITY INNOVATION ©2003
Advanced Techniques cont’d
• Automated response (Boeing)
– Integrates firewall, intrusion detection, filtering router, and
network management technologies
– Local intrusion detectors determines threat presence
– Firewalls communicate intrusion detection information to
each other
– Firewalls cooperate to locate the intruder
– Network managers automatically reconfigure the network to
thwart the attack
– Firewalls and filtering routers dynamically alter filtering
rules to block the intruder
– Dynamic reconfiguration of logging, monitoring, and access
control in response to detected suspicious activity
– "Fusion" of intrusion-detection data reported by different
detectors
– The monitoring is also adapted as part of the response, to
help pinpoint the problem and its source
72
SECURITY INNOVATION ©2003
Advanced Techniques cont’d
• Survivable Active Networks (Bellcore)
– Will allow highly configurable network elements to cooperate
with networked hosts to detect, isolate, and recover quickly
and automatically from damage due to errors or malicious
attacks
– "Ablative software" will allow suspect activity to be "peeled
off" the system while continuing to operate in a
microenvironment
• Planning and procedural reasoning (SRI)
– Suggest and implement incident recovery procedures
– Uses AI-based automated planning technology for both
analysis and recovery and repair
– Generates explanations to help the sys admin understand what
happened and what to do about it
– Integrate intrusion response tools, to combine the functionality
of many tools that specialize in particular areas of incident
management, into a security anchor desk (USC-ISI)
73
SECURITY INNOVATION ©2003
Open Questions
• Detection performance in realistic settings with single
methods and combinations of methods
• Detection performance with faulty and missing data
• False positive and false negative rates
• Time to detection
• Scalability
• Dependence on good intruder models
• Distinction from common failure modes
• What data to collect/observe
74
SECURITY INNOVATION ©2003
Common Intrusion Detection
Framework
• Standard Interfaces
E1
E2
– an interconnection
framework for data
collection, analysis, and
response components
– extensible architecture
– reuse of core technology
– facilitate tech transfer
– reduce cost
E3
A1
C
A2
D
75
E
A
D
C
Standard API
Event Generator
Event Analyzer
Event Database
System-specific Controller
SECURITY INNOVATION ©2003
Strategic Intrusion Assessment
• In a two-week period, AFIWC’s intrusion detectors at 100 AFBs
alarmed on 2 million sessions
• After manual review, these were reduced to 12,000 suspicious events
• After further manual review, these were reduced to four actual
incidents
National
Reporting Centers
Regional Reporting
Centers (CERTs)
DoD Reporting
Centers
Organizational
Security Centers
International/Allied
Reporting Centers
Correlation
Patterns
Classification
Infer intent
Assess damage
Predict future status
Assess certainty
Local Intrusion
Detectors
76
• Most alarms are false positives
• Most true positives are trivial incidents
• Of the significant incidents, most are isolated attacks to be dealt
with locally
SECURITY INNOVATION ©2003
Strategic Intrusion Assessment
Suppress false alarms
• Peer-to-peer cooperation
among detectors to decide
what to report to higher
levels.
• Detectors must be able to:
Correlate & infer intent
• Plan recognition
– discover each other
– negotiate requirements
– collaborate on
diagnosis/response
• Improve individual detectors
– Distinguish what is trivial
from significant
– Distinguish what is locally
relevant
77
SECURITY INNOVATION ©2003
– Hypothesize goals for IW
adversaries
– Develop plans for
accomplishing each goal
• automated planning
technology
– Overlay with observed
incident data to discover
intent
• plan recognition
technology
– Estimate certainty
Security Detection and Response
Center
• Functions:
• Detection: Analyzes and
filters events reported from
lower layers
Assessment
–
•
for items of interest to this
layer, and
– for reporting to higher
layers
Assessment: to understand
coordinated events
–
–
of interest at this layer, and
for reporting to higher
layers
Tracing (e.g., IDIP, active
nets)
• Automated response (e.g.,
IDIP for connection
closing/filtering)
• Event notification
Tracing
Detection
Notification
Response
•
78
Significant investment
Early speculative investigations
No research
SECURITY INNOVATION ©2003
Conclusions
• Currently available technology is not
adequate for the problem
• Promising methods under investigation
show significant improvement over
current technology
• There is still a lot more to be done
79
SECURITY INNOVATION ©2003