Intrusion Detection Systems in the
University:
Methods and Issues
Leigh Heyman - Artificial Intelligence Lab
Massachusetts Institute of Technology
leigh@ai.mit.edu
(with assistance from Curt Freeland, Univ. of Notre Dame and Sally
Goldberg, St. Mary's College )
Leigh Heyman,GCIA
Artificial Intelligence L
Introduction
– Mix of general and technical
– "sureveillance cameras"
– A quick (and surprising) statistic
What is Intrusion Detection?
– Host Based
– Network Based (Focus is on Network Systems)
Methods
– Implementation
– IDSes-- discussion of free software systems
"
"
SHADOW
Snort
Issues
– Politics
Privacy concerns
Access to data
Evidence: Chain of Custody
Network Security BIG PICTURE
" Policy is key
Why run an IDS?
"
–
–
–
–
Summary
Leigh Heyman,GCIA
– an intrusion
Artificial Intelligence L
What's this???
"
"
"
Leigh Heyman,GCIA
Video cameras for
surveillance
Established method of
crime
prevention/prosecution
Can we do the same with
our network?
Artificial Intelligence L
Do you know what's on your net?
Seen this in your syslogs?:
Aug 13 07:01:58 rpc.statd[315]: gethostbyname error
for^X<F7><FF><BF>^X<F7><FF><BF>^Y<F7><FF><BF>^Y<F7><FF><BF>^Z
<F7><FF><F>^Z<F7><FF><BF>^[<F7><FF><BF>^[<F7><FF><BF>%8x%8x%
8x%8x%8x%8x%8x%8x%8x%236x%n%137x%n%10x%n%192x%n<90><90><
90><90><90><90>
Statistics from our IDS indicate on average one "network
event" EVERY THREE HOURS!
("Network Events" are any data on the network which, according to
your policy, may indicate malicious activity, including:
port/host scanning, vulnerability scanning, "out-of-spec" packets-SYNFIN, tiny fragaments,etc., active exploits, DNS zone transfer
attempts and more...)
Leigh Heyman,GCIA
Artificial Intelligence L
What is Intrusion Detection?
Intrustion detection systems monitor and
analyze key system "vital signs" to detect
and track certain types of security policy
violations
(ID, and IDS refer to Intrusion Detection, and Intrusion Detection
System(s) respectively)
Leigh Heyman,GCIA
Artificial Intelligence L
Host Based ID
"
Watches individual systems
–
System logs
"
syslog
"
sulog, etc.
–
Personal firewalls
• Norton, McAfee, NetworkICE, etc.
–
Tripwire
–
Tcp-wrappers
–
Portsentry
Leigh Heyman,GCIA
Artificial Intelligence L
Network Based ID
"
Network ID systems capture all the data from the
network stream and process that data against a set of
signatures, or patterns of known attacks.
"
Centralized data about entire network of hosts
"
essentially a network sniffer.
– plugs into a shared bandwidth link on the network.
– “
sees”all packets that go by on the wire.
– can save copies of the entire packet, the packet headers, or just
those packets which meet certain specifications, to the system
disk.
– analysis determines what is happening on the network.
Leigh Heyman,GCIA
Artificial Intelligence L
Methods
Leigh Heyman,GCIA
Artificial Intelligence L
Implementing a NIDS
"
What makes a good IDS?
– First and foremost is log fidelity- is all the info there?
– "Tunability"-- the ability to write/modify the attack signature
filters
– Low "false-positive/negative" rate, and the abiity to tune to
optimize this rate
– A “friendly”user interface requires less skill and knowledge to
operate
– The capability to filter the input-- and control the input filter
• Cannot truly watch and analyze every packet, so we want
the ability to decide which ones to watch
– archiving/database features
– Cheap is always good.
Leigh Heyman,GCIA
Artificial Intelligence L
Implementing a NIDS
"
Leigh Heyman,GCIA
Where does it go?
–
DMZ model: Inside or outside the DMZ
often a matter of religion, truly robust
IDSes will have sensors on both sides.
–
With most routers and some switches
you have the ability to "mirror" or "span"
a port, the sensor can be on that port,
this is better than putting in "in the
stream"
–
Be careful with switches, often
"mirroring" or "spanning" can have
adverse affects on switch performance.
–
Few IDSes can keep up with ethernet
speeds, anything beyond 300Mbps is
pushing it.
Artificial Intelligence L
Implementing a NIDS
• If you haves a single Internet connection, you could get by with a
single sensor to monitor that feed.
• Sites with multiple external connections or a single high-speed
(ethernet) uplink will require multiple sensors!
• Residence Hall networks
– Prime location of unsecured, unmaintained systems
– Most common scenario: student wkstation compromised, then used to
attack others
• Business unit networks (bookstore, athletic ticket office, cafeteria)
– Any place where money is involved....
• Corporate Records (grades, registration, accounts due)
– Any place where protection of sensitive information is a priority...
• "mobile sensor" to plug in to hot-spots as needed.
Leigh Heyman,GCIA
Artificial Intelligence L
Implementing a NIDS
• care and feeding.
– keep the IDS systems secure!
– Run the system for ~1 month before really taking action
– bitsy.mit.edu
– go through the output regularly.
• analyst may spend as much as 20 minutes per hour on
analyzed output from a single sensor.
• Multiple sensor sites will require more time to examine and
correlate.
– monitor disk space consumption.
– back up the collected data in a manner consistent with the
accepted rules of criminal evidence collection.
Leigh Heyman,GCIA
Artificial Intelligence L
IDSes
"
Free
–
SHADOW
"
–
Snort
"
"
One of the first broadly implemented free IDSes. VERY high
fidelity output, but requires a lot of knowledge to operate.
Full featured, very robust... becoming the de-facto free IDS
Commercial
–
Network Flight Recorder
"
–
Popular product, as simple or complex as you want it to be.
"Appliance" type implementation. Research/Academic licensed
version
RealSecure
"
Leigh Heyman,GCIA
Robust, modular implementation. NIDS is a component of larger
security suite including host-based ID, and firewalling
Artificial Intelligence L
SHADOW
"
By default, Shadow collects everything on the wire.
"
The output can consume extraordinary amounts of disk.
"
Comes with a built in web-based analysis tool.
"
Analysis tool handles multiple sensors.
"
Drawbacks:
–
output requires strong knowledge to interpret.
–
output is not flashy.
–
not being (actively) developed.
–
Analyst station needs a fast CPU, and a lot of memory!
Leigh Heyman,GCIA
Artificial Intelligence L
SHADOW
"
Perl scripts with tcpdump
–
(at least) two PCs:
"
Sensor node(s)
–
"
Analysis host
–
Leigh Heyman,GCIA
runs tcpdump with rudimentary filter to hourly log file
hourly cron job pulls tcpdump log from sensor, and
processes it against more complex series of filters. Perl
scripts parse through the output, finds scans, then generates
html output.
Artificial Intelligence L
Leigh Heyman,GCIA
Artificial Intelligence L
Snort
"
simple, efficient FREE IDS
"
Very well-written and maintaned, robust application
"
Snort is “driven”by a set of (community developed) rules.
"
Actively (constantly) under development.
"
Windows and UNIX versions available (source form).
"
"
SnortSnarf (an add-on) provides a web-based analysis
tool.
In general requires lower level of skill to operate and
analyze output
Leigh Heyman,GCIA
Artificial Intelligence L
Snort
"
"
"
"
"
Alerts generated and/or packets logged when a "rule"
is triggered.
Very simple rule language for writing your own rules
Ability to log alerts to syslog, directories in ascii,
tcpdump format raw data, even send "windowgrams"
Different alert styles from one-line, to verbose
Modular "plug-in" architechture for adding functionality
– Many available plug-ins, including SQL and Oracle database
logging, statistical analysis, TCP stream and telnet session
reassembly, active response using "sniping"
"
Resistant against some of the newer attacks directed at
foiling IDSes
Leigh Heyman,GCIA
Artificial Intelligence L
Leigh Heyman,GCIA
Artificial Intelligence L
Issues
Leigh Heyman,GCIA
Artificial Intelligence L
Political Issues
"
Privacy Concerns
–
No easy answers here
–
Much of the responsibilty lies with the analyst
–
Can only manipulate the data so much before it loses its
usefulness
–
Some talk about whether the network headers on certain
networks constitute "student information" under FERPA.
Still an open question.
Leigh Heyman,GCIA
Artificial Intelligence L
Politics
Privacy concerns (cont'd)
"
What Notre Dame did when the users found out about it,
and threated to stampede because they thought their
privacy was being invaded:
– Newspaper learned of IDS, and launched Carnivore-like hype
–
Showed them the IDS output!
"
"
"
"
Leigh Heyman,GCIA
made the IDS output available to anyone on campus and invited
all to explore it for themselves.
asked them to find the sensitive information (from their host) that
they are afraid was gathered.
showed them that the IDS just looks for “signatures”that should
not appear on the network.
assured them that an have not got the time, or the desire, to read
their mail, monitor their surfing, or peruse their files.
Artificial Intelligence L
Access to Data
"
So now that we are seeing results from our IDS, what
do we do with it, who do we show it to?
"cease and decist" notices
– C&D clearing houses:
–
"
Internet Storm Watch
Availability of web output
– Sanitization of data before publishing
– Who needs to know
– Staging the data (classification model)
–
"
"
–
Only ID analysts have access "classified" raw data, and analysis
host itself
"Declassified" data can then be propagated to interested party
via a more publicly available means
Why do it this way......
Leigh Heyman,GCIA
Artificial Intelligence L
Chain of Custody
"
"
The reason we do it this way
–
Must preserve "chain-of-custody" should the data ever be
needed as evidence
–
If everyone has login shells on the sensor and/or
analysis station, the integrity of the data as evidence is
lost
–
Devise methods to have an accurate and reliable audit of
access to the raw and post-processed data
Lastly, the more loudly we announce the presence of
our IDS, and its data, the more likely it will be viewed by
the "bad guys"... how hard is it to turn a tool into a
weapon?
Leigh Heyman,GCIA
Artificial Intelligence L
Network ID, the BIG PICTURE
"
IDS is only a tool
–
–
"
Only useful as a component of a larger "defense-indepth" framework.
–
"
"Detection" is not "Prevention"
The Intrusion Prevention System, by Brooklyn Bridge
Technologies inc.
Safe rating system: 60TLTR
" How long before the folks with the big guns show up?
Policy is key
–
–
Establish users' expectation of privacy
Cannot detect "network events"-- anomalous traffic-without at least loosly defined baseline
Leigh Heyman,GCIA
Artificial Intelligence L
Why Should we run an IDS?
"
Will an IDS help us block Napster/Gnutella/Cow Porn?
– No, but it will point out the depth of your problems!
– Yes, because now you know what is eating up all of your bandwidth.
"
Will an IDS make things simpler for the security staff?
– No, because they will now be aware of all of the nasty things on the
wire.
– Yes, because they can have specific data on where to focus the
needs and skills of the group.
"
Will an IDS create problems for the users?
– No, because the IDS process is invisible to the users.
– Yes, because the users may find certain IP/port combinations
blocked due to the information gleaned from the IDS.
Leigh Heyman,GCIA
Artificial Intelligence L
Summary
• Intrusion Detection systems can help you secure your
network
• give a clear picture of the security landscape
• help tune the security infrastructure in accordance
• significantly improve most elements of the incident
handling capability
– Preparation, detection, containment, and eradication
• IDSes require several investments:
• time required to actively monitor and analyze IDS output
• hardware for the sensor and analysis stations.
• staff training to make the most of the IDS system’s potential.
• You need to prepare for a surprise! .....
Leigh Heyman,GCIA
Artificial Intelligence L
An Incident....
Leigh Heyman,GCIA
Artificial Intelligence L
August
"
"
th
13
2000
52 hosts comprosmised in 20 seconds using scripted
rpc.statd attack
What the syslogs showed:
Aug13 07:01:58 txakoli syslogd: Cannot glue message parts together
Aug 13 07:01:59 txakoli 173>
Aug 13 07:01:58 rpc.statd[315]: gethostbyname error for
^X<F7><FF><BF>^X<F7><FF><BF>^Y<F7><FF><BF>^Y<F7><FF><BF>^Z<F7><FF><F
>^Z<F7><FF><BF>^[<F7><FF><BF>^[<F7><FF><BF>%8x%8x%8x%8x%8x%8x%8x%8
x%8x%236x%n%137x%n%10x%n%192x%n<90><90><90><90><90><90><90><90><90
><90><90><90><90><90><90><90><90><90><90><90><90><90><90><90><90><90><
90><90><90><90><90><90><90><90><90><90><90><90><90><90><90><90><90><90
> ....... etc.
"
"
So.... whodunit? What evidence do we have?
– Where did the attack come from?
– What was the attack method?
– Do we have any hope of catching the bad guy?
What if the attack script erased the logs?
Leigh Heyman,GCIA
Artificial Intelligence L
August
"
th
13
2000
What the network saw:
(names and addresses have been left unchanged to implicate the guilty)
07:01:51.141438 158.135.16.113.657 > 128.52.54.26.111: udp 56
07:01:51.141918 128.52.54.26.111 > 158.135.16.113.657: udp 28
07:01:55.217308 158.135.16.113.658 > 128.52.54.26.1011: udp 1076
07:02:02.230468 158.135.16.113.1517 > 128.52.54.26.39168 : S 843433918:843433918(0)
win 32120 (DF)
07:02:02.230706 128.52.54.26.39168 > 158.135.16.113.1517: S 4178397570:4178397570(0)
ack 843433919 win
07:02:02.288170 158.135.16.113.1517 > 128.52.54.26.39168: . ack 1 win 32120 (DF)
07:02:02.288183 158.135.16.113.1517 > 128.52.54.26.39168: P 1:72(71) ack 1 win 32120
(DF) **
07:02:02.288496 128.52.54.26.39168 > 158.135.16.113.1517: . ack 72 win 32120 (DF)
07:02:03.290337 158.135.16.113.1517 > 128.52.54.26.39168: P 72:100(28) ack 1 win 32120
(DF) ***
07:02:03.290496 158.135.16.113.1517 > 128.52.54.26.39168: F 100:100(0) ack 1 win 32120
(DF)
07:02:03.290698 128.52.54.26.39168 > 158.135.16.113.1517: . ack 101 win 32120 (DF)
"
A clear audit trail, AND circumstantial evidence we can
forward to Nenette.
Leigh Heyman,GCIA
Artificial Intelligence L
August 13th 2000
"
Result
–
All 50+ hosts contained within a few hours, eradicated
and back online in less than 24
–
Captured subsequent incriminating data of attacker trying
to come back to visit compromised host
–
Disapointed Nenette because we recovered so fast we
never broke the $8000 mark.
Leigh Heyman,GCIA
Artificial Intelligence L
"
References
URLs
– www.sans.org
– www.snort.org
– www.incidents.org (internet storm watch)
– www.whitehats.com
– www.securityfocus.com
– www.honeynet.org
"
Books
– Intrustion Detection, An Analyst's Handbook (2nd Edition), by
Stephen Northcutt et al.
"
Mailing Lists
– Many lists at /www.securityfocus.com :
"
"
"
Leigh Heyman,GCIA
bugtraq
focus-ids
focus-incidents
– digests and searcheable archives
Artificial Intelligence L
Thank You
leigh@ai.mit.edu
Leigh Heyman,GCIA
Artificial Intelligence L