IC3 - Network Security
An Introduction to Intrusion Detection
and Vulnerability Assessment
RHUL, 8-Dec-2003
Andreas Fuchsberger & Robert Christian, F.A.C.T.S. Group
1
Agenda
•
Basics & Definitions
•
Why Intrusion Detection and Vulnerability Assessment
–
–
–
–
•
Attack Development
Vulnerability Development
Hacker Strategy
Anatomy of a Hack
VA
– Software
– Services ( Audits)
– Web-Based Services
•
IDS
– Host based IDS
– Network Based IDS
•
•
Demo of VA and IDS
Current technological Approaches
– “Honey Pots”
– Appliances
•
Summary
– Critical Issues
2
Basic and Definitions
• Perimeter security devices (e.g. firewalls) and
computer security mechanisms (e.g.
application and OS security) can only prevent
attacks by outsiders.
• They may fail to do so: a firewall may be
misconfigured, a password may be sniffed off
the network, a new attack type may emerge.
• They do not detect when an attack is underway
or has taken place.
• And they do not react to attacks.
3
Basics and Definitions
• Example:
– Imagine continuous inspection of a Unix system by hand (similar
examples for NT, W2K):
– The following checklist is from CERT
(http://www.cert.org/tech_tips/intruder_detection_checklist.html):
1. Examine log files for connections from unusual locations or other
unusual activity. For example, look at your 'last' log, process
accounting, all logs created by syslog, and other security logs.
2. Look for setuid and setgid files (especially setuid root files)
everywhere on your system. Intruders often leave setuid copies of
/bin/sh or /bin/time around to allow them root access at a later time.
4
Ad Hoc Intrusion Detection
• Imagine the complexity and degree of expertise
needed to carry out the tasks in this checklist
for every host and every sensitive network link
on a network every single day.
• The ad hoc approach is not recommended!
• Automated systems are needed:
– monitor multiple hosts and network links for
suspicious behaviour;
– report this behaviour, possibly react to it.
• Hence: Intrusion Detection Systems (IDS).
5
Intrusion Detection Systems
• Popular second layer of technical Information Security
enforcement
• Passive supervision of exiting network, analogues to
intruder alarms
– Creates more work for personal
• There exist 2 different approaches to the
implementation of Intrusion Detection Systems (IDS)
– Knowledge-based IDS
• Network based
• Host based
– Behaviour-based IDS
• Statistical anomaly detection
7
Why Intrusion Detection and
Vulnerability Assessment
Auto
Coordinated
Cross site scripting
Attack
Sophistication
“stealth” / advanced scanning
techniques
High
packet spoofing
Staged
denial of service
distributed
attack tools
sniffers
sweepers
www attacks
automated probes/scans
GUI
back doors
network mgmt. diagnostics
disabling audits
burglaries
Attack
Sophistication
hijacking
sessions
exploiting known vulnerabilities
password cracking
self-replicating code
Intruder Knowledge
password guessing
Low
1980
1985
1990
1995
2000
Source: Carnegie Mellon University
8
Why Intrusion Detection
and Vulnerability Assessment
4 Vulnerability Development
700
600
Linux (aggr.)
500
Solaris
400
Windows NT
Gesamt
300
200
100
0
1997
1998
1999
Source: SecurityFocus
2000
(Cum.)
9
Why Intrusion Detection
and Vulnerability Assessment
Vulnerability & Exploit Lifecycle
Vulnerability Scanners
adding detection signature
Widespread
Awareness
First
Discovery
Selective
Awareness
Advisory Release
10
Why Intrusion Detection
and Vulnerability Assessment
Unauthorized Access to Networks
11
Why Intrusion Detection
and Vulnerability Assessment
Origin of the Attack
12
Why Intrusion Detection and
Vulnerability Assessment
Source of the Attack
13
Why Intrusion Detection and
Vulnerability Assessment
Which Type of Attacks ?
2001 CSI/FBI - Computer Crime and Security Survey
14
Why Intrusion Detection and
Vulnerability Assessment
Types of Attacks
15
Why Intrusion Detection and
Vulnerability Assessment
Reactions to attacks
16
Why Intrusion Detection and
Vulnerability Assessment
“Classic”
Hacker Strategy
17
Why Intrusion Detection and
Vulnerability Assessment
PING
CORP
SWEEP
NETWORK
Internet
Primary Target Identification - Identify Hosts (
) with external visibility
denotes internal hosts with high value data but no external view
18
Why Intrusion Detection and
Vulnerability Assessment
DNS
PORT
CORP
NFS
SWEEP
NETWORK
WEB
Primary Target Analysis - Identify services running on visible hosts
to prioritize further probing activities
19
Why Intrusion Detection and
Vulnerability Assessment
FINGER
NFS
CORP
NETWORK
Primary Target Selection - Determine vulnerability state of weakest point
and concentrate further activities against this system
20
Why Intrusion Detection and
Vulnerability Assessment
Rlogin Root
NFS
CORP
NETWORK
Primary Target Exploitation - Gain privileges & control of primary target
- attacker now controls a ‘trusted’ corporate system !
21
Why Intrusion Detection and
Vulnerability Assessment
R&D
$
NFS
CORP
NETWORK
HR
Secondary Target Identification - Probing for high value information or systems
which are then compromised and data stolen or trojan horses planted, etc.
22
Summary / Schematic
Big Widget’s Network
crack
Unix
Web
Server
netbus
NT
Unix
NT
Firewall
Router
Network
E-Mail
Server
imap
Clients & Workstations
23
Denial of Service
• Denial of Service attacks (DoS)
In contrast to unauthorised access attacks a
DoS attack does not need to contain method
for communicating back to the attacker
• Distributed Denial of Service (DDoS) attacks
– Trin00/Stacheldraht (Feb 2000)
• Attacks on ebay, amazon.com and etrade.com
– MS.Blaster (August 2003)
• Problem of lack of metrics to measure the
impact of Denial of Service attacks – more
research required
24
Vulnerability Assessment
• Vulnerability Assessment Methods
– Software solutions (ISS Scanner, Stat, Nessus etc.)
– Audit Services (manual Penetration tests etc)
– Web based commercial (Qualys, Security Point etc)
• Keep up-to-date with security (and other)
patches
– Form Microsoft OS www.windowsupdate.com
• Enterprise version available
– Microsoft Baseline Security Advisor
• Includes hfnetcheck.exe (from Shavlik)
– Similar for SUN, HP, IBM, CISCO etc. OS
25
Vulnerability Assessment (VA)
Vulnerability
Assessment
DEMO
26
Intrusion Detection
• Intrusion Detection Systems (IDS)
• Intrusion Prevention Systems (IPS)
27
Knowledge-based IDS
• ALL commercial IDS look for attack signatures:
– specific patterns of network traffic or activity in log
files that indicate suspicious behaviour.
• Called a knowledge-based or misuse detection
IDS
• Example signatures might include:
– a number of recent failed login attempts on a
sensitive host;
– a certain pattern of bits in an IP packet, indicating a
buffer overflow attack;
– certain types of TCP SYN packets, indicating a SYN
flood DoS attack.
28
Knowledge-based IDS
• Knowledge-based IDS uses information such as:
– Security policy;
– Known vulnerabilities of particular OS and applications;
– Known attacks on systems.
• They are only as good as the information in the
database of attack signatures:
– new vulnerabilities not in the database are constantly being
discovered and exploited;
– vendors need to keep up to date with latest attacks and issue
database updates; customers need to install these;
– large number of vulnerabilities and different exploitation
methods, so effective database difficult to build;
– large database makes IDS slow to use.
29
Behaviour-based IDS
• Statistical Anomaly Detection (or behaviourbased detection) is a methodology where
statistical techniques are used to detect
penetrations and attacks.
• Begin by establishing base-line statistical
behaviour: what is normal for this system?
• Then gather new statistical data and measure
the deviation from the base-line.
• If a threshold is exceeded, issue an alarm.
30
Behaviour-based IDS
• Example: monitor the number of failed login
attempts at a sensitive host over a period;
– if a burst of failures occurs, an attack may be under
way;
– or maybe the admin just forgot his password?
• This raises the issue of false positives (an
attack is flagged when one was not taking
place – a false alarm) and false negatives (an
attack was missed because it fell within the
bounds of normal behaviour).
• This issue does also apply to knowledge-based
systems.
31
Behaviour-based IDS
• IDS does not need to know about security
vulnerabilities in a particular system
– the base-line defines normality;
– don’t need to know the details of the construction of a buffer
overflow packet.
• Normal behaviour may overlap with forbidden
behaviour.
– Legitimate users may deviate from the baseline, causing false
positives (e.g. user goes on holiday, or works late in the office,
or forgets password, or starts to use new application).
– If the base-line is adjusted dynamically and automatically, a
patient attacker may be able to gradually shift the base-line
over time so that his attack does not generate an alarm.
32
Host-based and Network-based IDS
• When an IDS looks for attack signatures in
network traffic, it is called a network-based IDS
(NIDS).
• When an IDS looks for attack signatures in log
files of hosts, it is called a host-based IDS
(HIDS).
• Naturally, the most effective Intrusion Detection
System will make use of both kinds of
information.
33
IDS Architecture
• Distributed set of sensors – either located on
hosts or on network – to gather data.
• Centralised console to manage sensor
network, analyze data, report and react.
• Ideally:
– Protected communications between sensors and
console;
– Protected storage for signature database/logs;
– Secure console configuration;
– Secured signature updates from vendor;
– Otherwise, the IDS itself can be attacked and
manipulated.
34
Placement of Network-based IDS
Internet
Sensor
Mail server
Firewall
Perimeter
Network
Sensor
Web server
Sensor
Console
Protected Network
38
Host-based IDS
• Typically monitors system, event, and security
logs on Windows and syslog in Unix
environments.
• Checks key system files and executables via
checksums at regular intervals for unexpected
changes.
• Some products can use regular-expressions to
refine attack signatures (e.g. passwd program
executed AND .rhosts file changed).
• Some products listen to port activity and alert
when specific ports are accessed – limited
NIDS capability.
39
Placement of Host-based IDS
Internet
Firewall
Sensor
Mail server
Perimeter
Network
Web server
Sensor
Human
Resources
Network
Console
Sensor
42
IDS as a Response Tool
• Given the (near) real-time nature of IDS alerts,
an IDS can be used as a response tool as well
as for detection.
• NIDS and HIDS have different response
capabilities – because they detect different
attacks, or the same attacks but in different
ways.
43
HIDS and NIDS
• There are attack types that a HIDS can detect but a
NIDS cannot:
– SYN flood, Land, Smurf and Teardrop attacks, BackOrifice,…
• And vice-versa:
– Trojan login script, walk up to unattended keyboard attack,
encrypted traffic,…
• For more reliable detection, combine both types of IDS.
44
IDS Response Options
Notification
Network-based
Host-based
Alarm to console
Alarm to console
E-Mail notification
E-Mail notification
SNMP trap
SNMP trap
View active session
Storage
Log summary
Log summary
Log raw network data
Active
Kill connection (TCP
Reset)
Re-configure firewall
Terminate user login
Disable user account
Restore index.html
45
IDS Response Options
• Dangers of automated response:
– Attacker tricks IDS to respond, but response aimed
at innocent target (say, by spoofing source IP
address);
– Users locked out of their accounts because of false
positives;
– Repeated e-mail notification becomes a denial of
service attack on sysadmin’s e-mail account;
– Repeated restoration of index.html from CD reduces
website availability.
46
Intrusion Detection
Intrusion
Detection
DEMO
47
What is Snort?
• Snort is a fast, flexible, small-footprint, opensource NIDS developed by the security
community and a “benevolent dictator”
• Lead coder: Marty Roesch, now founder of
Sourcefire (www.sourcefire.com)
• Initially developed in late 1998 as a sniffer with
consistent output, unlike protocol-dependent
output of TCPDump
• Licensed under GPL, but version 2.0 may
change to a different license
48
Snort Rules
• Snort rules are extremely flexible and are easy
to modify, unlike many commercial NIDS
• Sample rule to detect SubSeven trojan:
alert tcp $EXTERNAL_NET 27374 -> $HOME_NET any (msg:"BACKDOOR
subseven 22"; flags: A+; content:
"|0d0a5b52504c5d3030320d0a|"; reference:arachnids,485;
reference:url,www.hackfix.org/subseven/; sid:103;
classtype:misc-activity; rev:4;)
• Elements before parentheses comprise ‘rule
header’
• Elements in parentheses are ‘rule options’
49
Third-Party Enhancements
• Analysis Console for Intrusion Databases
(ACID)
– http://acidlab.sourceforge.net/
– PHP-based analysis engine to search and process a
database of security events generated by various
IDSes, firewalls, and network monitoring tools
– Query-builder and search interface, packet viewer
(decoder), alert management, chart and statistics
generation
– Description and screenshots taken from ACID web
50
Third-Party Enhancements
• Demarc
– www.demarc.com
– NIDS management console, integrating Snort with
the convenience and power of a centralized
interface for all network sensors
– Monitor all servers / hosts to make sure network
services such as a mail or web servers remain
accessible at all times
– Monitor system logs for anomalous log entries that
may indicate intruders or system malfunctions
– Description and screenshots taken from demarc web
53
Intrusion Prevention System - IPS
• Relatively new (marketing) term
• Essentially a combination of access control
(firewall/router) and intrusion detection systems
– Often shared technologies between stateful
inspection and signature recognition (“looking deep
into the packet”)
– Inline network IDS allows for instant access control
policy modification
• Recent Gartner study claims by 2005 only
integrated firewalls with IDS (i.e. IPS) will
survive
• Most success to-date with “flood” attacks
57
Honeypots
• Technology used to track, learn and gather evidence of
hacker activities
• Definition
– “… a resource whose value is being attacked or compromised”
Laurence Spitzner, “The value of honeypots”, SecurityFocus, October 2001
• Strategically placed systems designed to mimic
production systems, but not reveal “real” data
• Modes of operation
–
–
–
–
Baiting
Waiting
Collating
Disseminating
58
Honeypot types of implementation
• Level of Involvement
– Low Involvement: Port Listeners
– Mid Involvement: Fake Daemons
– High Involvement: Real Services
• Risk increases with level of involvement
59
Honeynet
• Network of honeypots
• Supplemented by firewalls and intrusion
detection systems - Honeywall
• Advantages:
– “More realistic” environment
– Improved possibilities to collect data
60
Honeynet
61
Sebek
• Sebek is a data capture tool designed to
capture all of the attackers activities on a
honeypot, without the attacker knowing it.
• 2 components.
– Client that runs on the honeypots, its purpose is to
capture all of the attackers activities (keystrokes, file
uploads, passwords) then covertly send the data to
the server.
– Server which collects the data from the honeypots.
The server normally runs on the Honeywall gateway.
• Since the Sebek client runs as a kernel module
on the honeypots, it can capture all activity,
including encrypted, such as SSH, IPSec
62
Honeynet using a Honeywall
63
Lecture Summary
• Threats are both internal and external.
• Prevention, detection and reaction are needed
in combination.
• Intrusion detection systems are a very useful
second line of defence (in addition to firewalls
and other safeguards).
• IDS deployment, customisation and
management is generally not straightforward.
64
Lecture Summary
•
•
•
•
•
•
•
•
•
•
Critical Issues
Why detect, if it cannot be prevented ?
Technical limitations
What defines the quality of any IDS
Reliability (False Positives / False Negatives)
Reliabilty
Managebility
Implementation
“Is a Patch really a Patch ?”
What other means exist ?
65
Lecture Summary
•
•
•
•
•
•
•
•
What do you absolutely need to know:
What is IDS / VA ?
Different Types
How do they function
What are issues to be observed ?
What are limitations to IDS / VA
… and if you really want to be good:
What are critical issues and how could they be
overcome ?
66
IDS Further Reading
• Stallings Chapter 9, pp.292-303 (possibly too
much emphasis on statistical approach;
research-focussed rather than commercially
focussed).
• An article: “The future of IDS” by Matthew
Tanase at SecurityFocus.com:
– http://online.securityfocus.com/infocus/1518
• An evaluation of IDS products by Kathleen A.
Jackson:
– http://www.sekure.net/ids/00416750.pdf
67
Thank You !
68