CS158B
Network Management
Dr. Stamp, Spring, 2005
Project 1
October 24, 2005
Signature based and Anomaly based
Network Intrusion Detection
Group Members:
Stephen Loftus
Kent Ho
Thesis:
In the area of Network Intrusion Detection (NID) there are two major ways of
determining that there has been a breach of security, signature and anomaly based, of
which signature may be the more efficient and reliable way to manage your network
security.
Project purpose:
NID, Compare and contrast: Signature vs. Anomaly. In the area of Network Intrusion
detection a primary way of determining that there has been a breach of security is using
signature based NID, we will talk about why we believe the signature based NID may be
the better way to manage network security as well as talk about some general issues
concerning NIDS, such as network management issues, general terms and definitions
related to NID, and briefly talk about how to handle an intrusion.
Topics to be covered:






Introduce Network Intrusion detection and how it affects us.
Compare and contrast signature based NID verses anomaly based NID, give
examples and the behavior of both.
Observe some techniques used in NID and definitions of keywords used in NID.
Handling and responding to an intrusion incident.
Observe network management issues addressing network intrusion detection as
well as its limitations, and why we should use a network intrusion detection
system.
Write code to simulate a port snifter/ hacker, to simulate intrusion on a server
machine and detect a breach of security using Ethereal.
1. Introduction
One of the major considerations when designing a network is the
confidentiality, integrity, and availability of the data on the network. The optimal
thing would be to keep unauthorized persons out of your network but that is not
always possible due to the large multi access point networks found today.
Unauthorized network activity may come from outside the network or it could
originate from inside the network. Network intrusion detection looks to find
people who have already compromised the outer layer of security. Understanding
the benefits of network intrusion detection gives us the knowledge of how to
prepare ourselves against an attack. An intrusion on a network is costly and
sometimes embarrassing. An unauthorized intrusion may bring the network down
by using a Denial of Service attack, or might may allow them to read, write or
delete critical information that may lead to loss of data or incorrect data being
distributed. The legal cost and other expenses associated to the attack may have
been saved if an intrusion detection product was implemented sooner.
Understanding intrusion detection is helpful in selecting or framing an intrusion
detection system for their network. In the area of Network Intrusion Detection
(NID) there are two major ways of determining that there has been a breach of
security, signature and anomaly based, of which signature may be the more
efficient and reliable way to manage your network security.
1
2. Anomaly vs. Signature
Anomaly:
An anomaly by definition is “something that is unusual or different from what
is expected.” In a network, it is often difficult to determine what is normal and
what is not. This inability to deterministically identify whether the actions
happening on the network are authorized or unauthorized can make the
implementation of an anomaly based detection system very difficult. The system
needs to be configured so that the proper level of monitoring is done. This
difficulty can lead to the detection posting many false positives if set to
aggressive, or to not catching anything if set too lax. Once you have determined
that an anomaly is happening the anomaly should be added to the list of
“signatures”, an anomaly is no longer unknown once it has been seen and
recorded. Due to the systems inability to “learn,” much of what the system
determines is an anomaly is really nothing (false positives), it is very complex and
difficult to define what is “normal” and what is not.
 What is normal- normal behavior is different for every network. For
example, Bob goes to work at 9AM, first checks his Email then opens an
application to start work. At 11:30AM, Bob goes to lunch and comes back
at 12:30 and starts work on updating some data base application. All of
this is recorded in the company’s logs. If Bob does the same thing next
morning, in the eyes of anomaly detection, it would be seen as “normal”.
 What is abnormal- If Bob started to work at 2 AM, using different
applications on his computer, his work pattern would be seen as out of the
ordinary and thus it may set off an intrusion alert to the system. When that
happens, an anomaly record is generated.
 What is an anomaly record?
An anomaly record generates three components:
1. Event- the audit record is of activity is abnormal, access to
programs are not the same as users usually use.
2. Time stamp- the user is using the computer at different times than
usual, accessing files and using applications at non-routine time
intervals than what was recorded in the audit records.
3. Profile- ID inserts a key field for an activity profile, this key is
unique to the profile and so if an exact match of this profile key is
found, an update to the anomaly record is done.
Signature:
Signature based intrusion detection is based on “known patterns” of
detrimental activity. The set of known signatures usually comes with the IDS
package and new ones are always available for update. Many of the techniques
used for unauthorized access to the network resources have been around for a
long time and have a very specific and detectable signatures. The required
signatures are set up in rules lists that are user configurable. These rule sets are
very large and complex and there is a direct proportion to the amount of CPU
required versus the rule set. Another problem is when you need to update it, the
hackers can get the same updates as you and be ready to defeat the newer
2
signatures. The whole signature based NDIS is often trying to catch up to the
hackers, and prevent the newest flaws in Microsoft software. The anomaly based
NDIS are often the source of the newest signatures.
 How can signatures miss something? There are often anomalies where
there is no known signature, as such, signature based NID will not detect
it. For example, skunks have a white stripe, it is documented as having a
white stripe, so it is a signature. If someone paints it gray, then the attack
would go undetected.
 Low alarm rates- signature based NID have low alarm rates because all it
has to do is to look up the list of known signatures of attacks and if it finds
a match, the alarm goes off. The disadvantage to this is that it is actually
generating a “false negative”, where an attack is going undetected. This is
where an anomaly detection NID may catch it.
 Accuracy- Signature based NID are very accurate because it has a list of
known attack patterns to look for, it is easy for it to identify suspected
packets and thus it can detect 90+% of the attacks.
 Speed- signature NID systems are fast since they are only doing a
comparison between what they are seeing and a predetermined rule.
 Negatives- If someone develops a new attack, there will be no protection.
The signature based system is, “only as strong as its rule set”
3. Intrusion detection behavior and techniques





Use MAC address in Intrusion Detection (RMON1) – If there is a direct
connection to the system without going through a gateway, an attack can
be made using the MAC address.
Use network mapping in Intrusion Detection- used to collect information
about your network. Network mapping scans the network using tools like
nmap to exploit the network.
Data Mining- much like mining for valuable resources: oil , gold, etc, in
NID, data mining collects significant data patterns of a network for
analyzing an attack plan into the network. These attacks can be based on
when certain systems on performing maintenance
The “door knob” attack is where the intruder goes to each host and only
tries a couple of attempted logins before moving to the next host. This few
number of attempts will not cause the host to lock down, nor will it be
caught by most of the NIDS signatures since it is very common for a user
to enter the wrong password once or twice. The signature is based not on
the individual failures, but rather the fact that the same entity is trying to
access many different hosts.
Port Scan on network- prior to an attack, an attacker will research the
network to find a open port, at the same time, it is gathering information
about the network and making a blue print of the network, seeking a
desirable area for an attack. The port scan can be done in a number of
ways to mask its true signature. During this “attack” each of the TCP/IP
Socket requests from the attacker will result in a SYN-NAK. These
numerous SYN-NAK’s will alert the NDIS that there is an attack
underway.
3


Flooding ports- keeping ports busy so that it cannot serve any customers,
this is the deny of service attack (DoS). Using the TCP/IP protocol method
of establishing a connection. The SYN messages that are sent to a
computer are sent with spoofed return IP addresses. The target machine
does not ever get a response since the IP address it is using is wrong.
Eventually the buffer on the target machine will fill up and a buffer
overflow will occur. This buffer overflow may lead to the system crashing
or to some hostile code being executed on the target machine.
Buffer overflow- even though buffer overflows are a part of almost all
software, they are a very well known and used technique to gain access to
systems. The attacker will send a message with a different size package
than what is expected. A memory space that was only intended to hold a
certain amount of data has more put into it. This extra code is the
malicious payload that the hacker is trying to get through.
4. Management issues





Outsourcing monitoring Pros:
o Staff shortage- who will be the IDS guy on the side? We are all
so busy with our own jobs, and even if we do it ourselves, we
are limited to how much time to spend looking at firewall logs.
If we hire outside company to do this, they can monitor the
system 24/7.
o Skill level- They are better at it since they do it all day.
 Cons:
o Price- can be costly depending on what you want.
o Can we trust them?
Convincing boss to pay for NIDS- bottom line- would need to show that
the benefits of IDS outweighs the cost of an attack resulting in loss/ stolen
data, bad reputation, legal cost, recovery cost, and embarrassments.
MSSP (Managed Security Service Provider)- MSSP has access to
resources that otherwise we could not afford, they are proficient in what
we lack in detection skills. Paying a fee for this service could provide us
with piece of mind and savings in time and money that otherwise our
company would spend to hire our own.
Limitations- the greatest limitation of intrusion detection is that there is
constantly new development. New ways to break into a system. It is very
difficult to prevent something we do not know anything about.
o Money- while good NIDS such as SNORT is free, some IDS may be
very costly.
o Hardware- unlike software, there is no update except to throw it out
and get a new one.
o Software- hard to write code to prevent what we do not know will
happen.
Real time NIDS is expensive, however it is cheaper in the long run
because you did not have to deal with an intrusion and its associated cost.
Checking after the fact is often easier, but at that point the damage is done.
4
5. Attack example:
Using three machines, a simulation of an intrusion is demonstrated.
1. Attacker machine: responsible for finding an open port then sending a file,
where the file could be for the benefit or the attacker. The attacker
machine will use java code “PortSniffer.java” to find a open port and send
a file or message to the server (who is being compromised).
2. Server: a machine that will be used for the simulation of an intrusion. The
server will have no protection and fully exposed to an attack.
3. Monitor: responsible for monitoring the attack, using Ethereal, it will see
and record the attack by the attacker machine on the server.
Conclusion:
A signature based NIDS catches 90+% of known documented attacks it is accurate
because it only has to look it up and it produces very little “false alarms”, however, it lets
a lot of real attacks go undetected because the attack pattern used is not on the “black
list”, thus, it is not a match. Here is where an Anomaly based NIDS may come in handy.
Anomaly based NIDS catches too much, most of what it sees as an “attack” is nothing
at all. It takes up a lot of resource because of false alarms. We had to design a filter just
so we only get “urgent” attacks picked up by an anomaly based NIDS. It is not accurate
and it is hard to implement because hackers are constantly evolving.
The combination of Signature based and Anomaly based intrusion detection systems
allows the network manager the best of both worlds, the result is an more effective and
more efficient network intrusion detection system that offers layered protection.
5
Reference:
http://www.webopedia.com/
http://www.sans.org/resources/idfaq/
http://www.bluekestrel.com/network_intrusion_detection_systems.htm
Kemmerer, Richard and Vigna, Giovanni, Intrusion Detection: A Brief History and
Overview, 2002.
Lee, Wenke and Stolfo, Salvatore, A Framework for constructiong Features and Models
for Intrusion Detection System, Columbia University.
Denning, Dorthy, An Intrusion-Detection Model. SRI International.
6