Real-Time Intrusion Detection with
Emphasis on Insider Attacks
Shambhu Upadhyaya
Computer Science and Engineering
University at Buffalo
Polytechnic University
October 3, 2003
Some Facts & Figures (CSI/FBI 03)
CEISARE @
2
Source of Attack
CEISARE @
3
Attack Types
CEISARE @
4
Actions Taken
CEISARE @
5
What Could be Learned From It?
 Good prevention techniques must be in place
 Good policies must be set up
 Need to know what is important
 Need to know the application environment
 IDS is a must
 But there is no IDS that is applicable to all environments
CEISARE @
6
Outline of the Talk
 General introduction
 Evolution of IDS
 Major players
 Insider threats and how to mitigate?
 Conclusion
CEISARE @
7
Outline of the Talk
 General introduction
 Evolution of IDS
 Major players
 Insider Threats and how to mitigate?
 Conclusion
CEISARE @
8
What is an IDS?
 In its general sense –
 Acquires information about its environment to analyze
system behavior
 Aims to discover security breaches, attempted breaches,
open vulnerabilities that could lead to potential breaches
 Types of information –
 Long term info. – a knowledge base of attacks (static)
 Configuration info. – a model of the current state (static)
 Audit info. – describing the events happening (dynamic)
CEISARE @
9
IDS Architecture
(Macroscopic View)
Slide adopted from UCDavis, Jeff Rowe
CEISARE @
10
IDS Side-effects
 False negatives (failed detection)
 poor coverage
 False positives (wrong indictment)
 poor QOS
 Degrade normal operation
 poor performance
CEISARE @
11
Outline of the Talk
 General introduction
 Evolution of IDS
 Major players
 Insider Threats and how to mitigate?
 Conclusion
CEISARE @
12
Evolution of IDS
Paul Innella’s timeline:
CEISARE @
13
Current State-of-the-art
 1st generation tools are largely signature based
 Security is by penetrate and patch
 Today’s focus is on detecting novel intrusions
 New techniques must consider insider attacks, social
engineering based break-ins etc.,
 Need for new paradigms – Design for Security?
 New ideas –
 Combining IDS with vulnerability analysis
 Detection is not fool-proof; must be merged with recovery
CEISARE @
14
Outline of the Talk
 General introduction
 Evolution of IDS
 Major players
 Insider Threats and how to mitigate?
 Conclusion
CEISARE @
15
Major Players – Academia
 Purdue –
 CERIAS
 UC Davis –
 Developed GrIDS (Graph based IDS)
 CMU – Home of CERT/CC
 Cornell
 Language-based security
 Columbia
 IDS and Data mining
 Above list is incomplete
CEISARE @
16
Major Players – Industries
 IBM Watson
 Global Security Analysis Laboratory
 Microsoft
 Started the Trustworthy Computing initiative in 2002
 Cisco
 Does research and development
 Builds intrusion detection appliances – sensors and software
 MAFTIA
 European Union of academia and industries
CEISARE @
17
Major Players –
Labs/Government
 SRI International –
 Developer of EMERALD through funds from ITO, DARPA
 Air Force Research Lab –
 Defensive Information Warfare Branch
 Naval Research Lab –
 Center for High Assurance Computer Systems
 Multi-level security
 National Institute of Standards and Technology –
 Computer Security Resource Center
 National Security Agency –
 Research and education
CEISARE @
18
Popular Websites
 SANS (System Administration, Networking and Security) Institute
 http://www.sans.org/aboutsans.php
 CERT/CC
 http://www.cert.org/
 CERIAS (Center for Education and Research in Information
Assurance and Security)
 http://www.cerias.purdue.edu/
 NIST (National Institute of Standards and Tech.)
 http://csrc.nist.gov/index.html
CEISARE @
19
IDS Tools List
 Mike Sobirey (copyright: Dr. Michael Sobirey)
 List of ID Tools from 1995-2000
 92 host- and network based Intrusion Detection (&
Response) Systems
 Additions are appreciated
 NIST Intrusion Detection Tools
 Coverage is only up to 1996 (not up-to-date)
 About 20+ tools listed
 The above two lists have little overlap (cover >110)
CEISARE @
20
Recent Releases

Responsible for real-time packet capture and analysis
(http://www.prelude-ids.org/) on Linux/Unix
 Prelude platforms
 Portsentry – An IDS that detects and responds to port scans against
a target host in real-time (http://www.psionic.com/products/)
 SPADE – Statistical Packet Anomaly Detection Engine
(http://www.silicondefense.com/) inspects recorded data for
anomalous behavior based on a computer score

Stealthwatch (Lancope), Stormwatch (Okena)

Stackguard – Protects from stack smashing attacks
(http://www.cse.ogi.edu/DISC/projects/immunix/StackGuard/)

Netscreen -- http://www.netscreen.com/products/

There is no tool that is universally applicable
CEISARE @
21
Outline of the Talk
 General introduction
 Evolution of IDS
 Major players
 Insider Threats and how to mitigate?
 Conclusion
CEISARE @
22
Who is an Insider?
CEISARE @
23
How to Deal with the Problem?
Model the Insider
Prevention of Insider Misuse
Detection, Analysis and Identification of Misuse
We focus on the detection only
CEISARE @
24
IDS with Emphasis on Insider
 Current systems are signature-based and they
use audit-trail or rule-based protection
 Not effective for insider attack detection
 Anomaly detection is applicable, but not very
effective
 New theory needed, proactive mechanisms
needed
CEISARE @
25
Guidelines for Effective Anomaly
Detection
 Use the principle of least privilege to achieve better
security
 Use mandatory access control wherever appropriate
 Data used for intrusion detection should be kept
simple and small
 Intrusion detection capabilities are enhanced if
environment specific factors are taken into account
CEISARE @
26
Our Approach

An out-of-the-box Reasoning Framework for intrusion
detection

Technique used:

Control flow checking from FT (basis for encapsulation of
owner’s intent)

Reasoning based on Theory of Risk Analysis from
Economics

Problem is similar to Pricing Under Uncertainty

S. Upadhyaya, R. Chinchani, K. Kwiat, “An Analytical Framework for Reasoning about Intrusions”,
IEEE SRDS 2001
CEISARE @
27
User Intent Encapsulation
 Obtain the intent of the
user either by inference
or query
 Session scope serves as
a certificate
 Reduces the search
space during monitoring
CEISARE @
28
Illustration of Search Space
Reduction
Commands
and System
calls
User
Audit
data
reduces
as we go
higher up
Kernel
Resources
CEISARE @
Typical
audit
data
29
Overall Layout of System
Operation
Disk
User
Sequence of
Operations
Interface
Network
Resource
CPU
Memory
System
CEISARE @
30
Expected Sequences
 Certain “normal” ways of doing a job
 Also, certain “less normal” ways of doing them
 A job is completed by performing a sequence
of operations
 May not be possible to enumerate all the
sequences
CEISARE @
31
Cost Analysis
 Cost of Operation
= Co
 Proportional to the amount of resources used
 Cost of Sequence
= Cd
 Proportional to the difference between current
chosen operation and past history
 Cost of Job
CEISARE @
= *Co + *Cd
32
Job Activity Stochastic
 At any stage, a user “chooses an operation”
with a probability
 “Choice of an operation” is a random variable
 Sequences of operations construct a discrete
stochastic process
CEISARE @
33
User Activity as a Martingale
 Theorem:
Let the lateral sequence of random
variables for any state i of a sequence of
operations be denoted as:
X1(ti, ), X2(ti, ), … Xn(ti, )
Such a sequence of user activity is a
Martingale
CEISARE @
34
An Example
nfrm
pine
exit
nop
nop
pine
ls
exit
nop
nop
mail
finger
nfrm
pine
exit
nfrm
pine
finger
exit
nop
(nfrm, pine, ls, mail, exit, finger, nop)
CEISARE @
A Note on Martingale

Martingale uses concepts of conditional probability and
has applications in economics

Model is used to predict market parameters like a share of
a stock

Future price of a commodity depends only on the last
known distribution and not on the entire history of the
prices

There is a parallel between uncertainties in intrusion
detection and the concept of pricing under uncertainty
CEISARE @
36
Reasoning
Tl
Nonintrusive
Nondeterministic
Th
Intrusive
Monotonically increasing costs
CEISARE @
37
Cost Scenarios
 Low Co + Low Cd
 Non-intrusive
 Maps into the non-intrusive region
 High Co + Low Cd
 Intrusive and tending toward a DoS attack on
resources
 Maps into the non-deterministic region
CEISARE @
38
…contd.
 Low Co + High Cd
 The intruder??
 Maps into the non-deterministic region
 High Co + High Cd
 The clumsy attack
 Maps into the intrusive region
CEISARE @
39
Quantification of Thresholds
 Threshold Tl
 Minimum cost over longest sequence
 Threshold Th
 Maximum cost over shortest sequence
CEISARE @
40
Algorithm: INIT_DISTR
(Generates the initial distribution)
 Enumerate all possible
sequences
 Find the longest sequence
 Create a discrete stochastic
process
 Generate probabilities at
each stage and shape the
distribution
CEISARE @
41
Algorithm: MODIFY_DISTR
(Modifies the existing distribution)
 Check to see at each stage of the
sequence if the user is conforming to the
profile
 At the job termination, if the sequence is
not intrusive, update the frequency
distributions and probabilities
CEISARE @
42
Algorithm: DECIDE
(Makes a decision in the non-deterministic region)
 Calculate the longest sequence from
current stage to complete the job. Move Tl
to that position
 The window (Th – Tl) depends on the
gradient of the cost accumulated since
DECIDE was last invoked
CEISARE @
43
Sketch of the Overall Algorithm
User logs into the system
Chooses the job s/he wishes to perform
Check the size of the session scope
If too large,warn user
YES
User wants to change it
Launch inter work-space level monitor
Create workspaces for the jobs
Launch workspace level monitor thread per workspace
Launch command level monitor thread per command
Report command type
Authenticate command
Loop
Report object accessed
Monitor Command
CEISARE @
44
Preliminary Implementation
 Developed in Java on Solaris 2.8
 A university environment was simulated
 Monitoring at basic command level
 Limited sequence monitoring
 Not many scenarios
 Perhaps, not realistic for actual deployment
CEISARE @
45
Test Cases

User activity collected over two months

Test cases grouped into four categories

1-user, 1-user with multiple logins, multiple users, multiple users
with multiple logins


Two sets of experiments – worst case and average case

Legitimate and intrusive operations
32 attacks

Obvious ones such as transferring /etc/passwd files, exploiting
vulnerabilities such as rdist, perl 5.0.1

Subtle attacks similar to mimicry attacks
CEISARE @
46
Summary of Results
Summary
1 User, No Multiple Logins 1 User, With Multiple Logins 2 Users, No Multiple Logins 2 Users, With Multiple Logins
User
Detection
87.50%
78.60%
74.90%
91.90%
and
Latency
33.4
35
36.1
29
User False Positives
12.50%
21.40%
25.10%
8.10%
False Negatives
0%
0%
0%
0%
User
Detection
98%
89%
100%
94.70%
and
Latency
0
11
0
9.6
Intruder False Positives
0%
0%
0%
0%
False Negatives
2%
11%
0%
5.30%
Intruder
Detection
99%
100%
98.20%
100%
and
Latency
0.4
0.7
0.6
0.5
User False Positives
0%
0%
0%
0%
False Negatives
1.40%
0%
1.80%
0%
Intruder
Detection
56%
81.30%
77.40%
91.50%
and
Latency
15.9
14.8
17
27
Intruder False Positives
0%
0%
0%
0%
False Negatives
44%
18.70%
22.60%
8.50%
CEISARE @
47
Types of Detected Intrusions
It can detect internal attacks  A cracker logs in and executes commands
 Inadvertent operator faults
 Internal abuse
 External attacks  Masquerading
 Subversion attacks by presenting overly
permissive session-scope (penalty in terms of
reduced QoS)
CEISARE @
48
Undetected Intrusive Activity
It cannot contain or detect  External Denial of Service attacks
 Extremely low-level network based attacks
CEISARE @
49
Ongoing Research
 Addressing outstanding issues like
 State explosions due to partial orderings
 Scalability
 Values of α, β , ??
 A more realistic prototype implementation and
testing
 Project is currently funded by DARPA
CEISARE @
50
Concluding Remarks – Vision
 Insider threat is very much real
 Penetrate and Patch method is not adequate
 CMU and other studies show current IDS are not effective
 Anomaly detection schemes, that are environmentindependent may be in the focus
 Monitoring at user command level has distinct advantages
 Conceptually independent of systems and applications
 As the no. of threats grows, IDS will become a required
element of system security
CEISARE @
51
Concluding Remarks – Research
 IDS and vulnerability analysis
 Effective means of system evaluation
 Good metrics for performance, coverage etc.
 Return on investment studies
 Merge IDS with firewalls
 Merge IDS with recovery
 It is not possible to detect all intrusions
 Protection against unknown threats – Proactive mechanisms
 Rapid Incident Response
CEISARE @
52