Journal of Information Assurance and Security 5 (2010)
An Anomaly-Based Approach for
Intrusion Detection in Web Traffic
Carmen Torrano-Gimenez, Alejandro Perez-Villegas
and Gonzalo Alvarez
Instituto de Física Aplicada,
Consejo Superior de Investigaciones Científicas,
Madrid, Spain
Mike Hsiao 2010.06.11
References
• Carmen Torrano-Gimenez, Alejandro Perez-Villegas and Gonzalo Alvarez,
“An Anomaly-Based Approach for Intrusion Detection in Web Traffic,” in
Journal of Information Assurance and Security, vol. 5, 2010.
• C. Torrano-Gimenez, A. Perez-Villegas and G. Alvarez, “A Self-learning
Anomaly-Based Web Application Firewall,” in 2nd International Workshop
in Computational Intelligence in Security for Information Systems (CISIS 09),
vol. 63 of , 85-92, Springer-Verlag, 2009.
• A. Liu, Y. Yuan, D. Wijesekera, and A. Stavrou, “SQLProb: A Proxy-Based
Architecture toward Preventing SQL Injection Attacks,” in Proc. of the
2009 ACM Symposium on Applied Computing (SAC’09), 2009.
• Fredrik Valeur, Giovanni Vigna, Christopher Kruegel, and Engin Kirda, “An
Anomaly-Driven Reverse Proxy for Web Applications,” in Proc. of the
2006 ACM Symposium on Applied Computing (SAC’06), 2006.
2
An Anomaly-Based Approach for Intrusion Detection in Web Traffic
Outline
• Introduction
• Web Applications and Web Attacks
– Web Applications
– Web Attacks
– Web Vulnerabilities
• System Overview
– Architecture (WAF: Web Application Firewall)
– Normal Behavior Description
– Detection Process
• Experiments: Case Study (Web Shopping)
– XML/Training/Testing
– WAF Protection Mechanism
– Performance/Results
• Comments
3
Introduction
• Web applications handle large amounts of
sensitive data, which makes web applications
even more attractive for malicious users.
– Identity supplanting, sensitive data hijacking,
unauthorized information, web content modification,
command execution, etc.
• Conventional firewall (operating at network and
transport layers) are usually not enough to
protect against web-specific attacks.
– To be really effective, the detection is to be moved to
the application layer.
4
Traditional Firewall (Layer 3/4)
valuable server
Network layer (3)
E.g., IP
attacker
Packet Inspection!
Transport layer (4)
E.g., TCP, UDP
Application layer (7)
E.g., HTTP, FTP, PRC
5
Traditional Firewall (Layer 3/4)
Most of the IDS can inspect the
application layer messages, but
they basically fall into “misuse”
based category, which only capture
known attacks.
Traditional Firewall: it can inspect
messages and headers carrying in
layer 3 and 4.
Some firewall may extend its
capability to capture layer 2
information.
6
Traditional Firewall (Layer 3/4)
•
Netfilter/iptables (L3/L4)
–
–
•
Snort (L3/L4 + L7 signature)
–
–
•
iptables -A INPUT -p TCP -i $RED_DEV --dport 135 -s 0/0 -j DROP
Such rules can not distinguish attacks from norms. All network traffic to TCP port 135 will be dropped.
alert tcp $EXTERNAL_NET any -> $HOME_NET 135 (msg:"NETBIOS DCERPC ISystemActivator path
overflow attempt little endian unicode"; flow:to_server,established; content:"|05|"; within:1;
byte_test:1,&,16,3,relative; content:"|5C 00 5C 00|"; byte_test:4,>,256,-8,little,relative;
flowbits:isset,dce.isystemactivator.bind; classtype:attempted-admin; sid:2351; rev:10;)
Such rule is much more precise than the previous one, but it relies on “syntactic” analysis, not
“semantics” analysis.
Shield (L3/L4 + L7 Semantic signature, Vulnerability-based IDS)
–
These tools are
useful for detecting
known attacks, but they
simply block the malicious traffic.
They are simply temporary solutions.
The vulnerable software need to be fixed
as soon as possible. And also, these rules may
not be robust to detect the attack and its variants.
7
IDS – Intrusion Detection System
• Signature Detection System (Misuse)
– Negative approach
– Known attacks/exploitations
– String Matching Techniques
• Anomaly Detection System (Anomaly)
– Positive approach
– Normal/Common Behavior
– Irregular behavior will be tagged as intrusive
• Hybrid
8
Traditional IDS shortcoming
• Signature Detection System (Misuse)
– Fragmentation, pattern changing, …
– False positive/negative problem
• Anomaly Detection System (Anomaly)
– Complex environments (large network with
multiple servers and operating systems)
• Up-to-date normal? Feasible normal?
– FP/FN problem
9
UTM - Unified Threat Management (2003)
• UTM是利用單一或簡單的介面設定來管控以及保護公司網路進出的安
全的閘道式設備。
• UTM依照不同機型，可以將以下幾種主要的功能選擇幾項整合於單一
主機中，路由、防火牆、垃圾郵件過濾、防毒（包含病毒、惡意軟體、
網路釣魚等...）、IDS或IPS（入侵偵測或防禦系統）、網頁過濾、
proxy 、VPN 、VOIP 、NAT、抵禦 DoS / DDoS (阻斷服務/分散式阻斷服
務)。
• UTM systems must
–
–
–
–
Be an appliance
Include multiple security features
Have a hardened OS
Be able to perform:
• Network firewalling
• Intrusion prevention (IPS)
• Gateway anti-virus
10
WAF – Web Application Firewall
• WAF analyzes the HTTP traffic (application
layer) in order to detect malicious behaviors
that can compromise the security of web
application.
• This paper relies on an XML file to describe
what a normal web application is.
11
Web Applications and Vulnerabilities
• Application
– Presentation, application logic, storage (see next page)
• IIS/Apache, Tomcat, MSSQL/MySQL
– Web content are dynamic
• CGI in Perl, Python, C/C++; JSP, PHP, ASP; Java, VB, C#
• Attack
– Static attack looks for security vulnerabilities in the web
application platform: web server, application server,
database server, firewall, OS, and third-party component
such as shopping cart, crypto modules, …
– Dynamic web attacks only request legal pages of the
application but they subvert the expected parameters.
• Vulnerability – OWASP Top 10
12
How WAF works?
Web Server
User
App Server
DB Server
Attacker
Media Server
13
Application Security Risk
OWASP (The Open Web Application Security Project)
OWASP Top 10 – 2010 (rc1)
14
2007 vs. 2009
15
Architecture
(reverse)
ModSecurity is a popular open source signature-based WAF.
16
Armorize SmartWAFTM
17
Reverse Proxy (+ Load Balance)
Web Server
Cash Flow 1
Internet
Cash Flow 2
WAF
WebMail
WAF 代替網頁伺服器回應 response:
可在收到 request 時, 檢查內容,
若正常, 則向網頁伺服器請求內容, 並回應.
But sometimes WAF is not enough.
Media Server
18
[*] Fredrik Valeur, Giovanni Vigna, Christopher Kruegel, and Engin Kirda, “An Anomaly-Driven Reverse Proxy
for Web Applications,” in Proc. of the 2006 ACM Symposium on Applied Computing (SAC’06), 2006.
Web Site Design
f1: function 1
X: table X
(a) an e-commerce web site
implemented with a single
server that relies on a single
back-end database and that
accesses a credit card
processing server.
(c) The database is modified to create
two different users u1 and u2, where u1
is allowed to access table x only and u2 is
able to access both table x and table y.
User u1 is associated with server B and
user u2 is associated with server C. 19
[*] Fredrik Valeur, Giovanni Vigna, Christopher Kruegel, and Engin Kirda, “An Anomaly-Driven Reverse Proxy
for Web Applications,” in Proc. of the 2006 ACM Symposium on Applied Computing (SAC’06), 2006.
• A web site could be made more resilient to attacks if it would be
possible to design both the server and the database infrastructure
so that different levels of access to the database and the hosts
running the server processes could be clearly enforced.
• Design (b)
– (i) non-sensitive, static information about the e-commerce company
(e.g., company contacts and support information) is accessible
through one server;
– (ii) the non-sensitive, dynamic information about product availability is
accessible through a second server that accesses a product database;
and, finally,
– (iii) the sensitive information about users is accessible through a third
server that relies on a user database, which is separated from the
product database.
– This last server has also access to the credit card processing server.
20
Normal Behavior Description
• The XML file contains rules regarding to the correctness of HTTP verbs,
HTTP headers, accessed resources (files), arguments, and values for the
arguments.
• Verbs.
– The verbs node simply specifies the list of allowed HTTP verbs. Requests using
any other verb will be rejected.
• Headers.
– The headers node specifies a list of some HTTP headers and their allowed
values. Different values will not be accepted.
• Directories.
– Each directory in the web application space is represented in the XML file by a
directory node.
– Each file in the web application space is represented by a file node.
– Input arguments are represented by argument nodes within the
corresponding file node.
• Legal values for arguments should meet some statistical rules.
21
The XML file is generated by
training/testing method.
22
Example of rules: Prefix
Prefix
/taiwan/content/imageView\.asp
/C2M21/manager_citation(_acts)?\.php.*
/F/[A-Z0-9]{51}\-[0-9]{5}
/cgi/openfile{3_0}?
(/[\w\-]*)*/hypage\.cgi
/ttscgi/ttsweb([0-9]|new)?
/cgi-bin/(?.Count.cgi|counter)
(/cgi-bin/file-upload\.cgi|eduArea/|cgin1110\.asp)
/saweb/pc|f\.file.*\.[Pp][Dd][Ff]
Snort rule (Blaster):
alert tcp $EXTERNAL_NET any -> $HOME_NET 135 (msg:“NETBIOS DCERPC ISystemActivator path overflow
attempt little endian unicode”; flow:to_server,established; content:“|05|”; within:1;
byte_test:1,&,16,3,relative; content:“|5C 00 5C 00|”; byte_test:4,>,256,-8,little,relative;
flowbits:isset,dce.isystemactivator.bind; classtype:attempted-admin; sid:2351; rev:10;)
23
Example of rules in Armorize
SmartWAF
All Regular Expression!!
24
WAF vs. Fortify RTA
Fortify RTA
WAF
25
A. Liu, Y. Yuan, D. Wijesekera, and A. Stavrou, “SQLProb: A Proxy-Based Architecture
toward Preventing SQL Injection Attacks,” in Proc. of the 2009 ACM Symposium on
Applied Computing (SAC’09), 2009.
26
Comments
• In order to achieve the deeper packet
inspection, the proxy design is used to
mitigate the effort in message extraction.
• Input validation is important task all kind of
web applications.
• Proxy (in front of different application server)
can focus on only checking the attacks that
related to the server.
• Misuse or anomaly approach?
27