Business Logic Bots: Friend or
Foe?
Amichai Shulman, CTO, Imperva
Ofer Shezaf, Leader, OWASP Israel
OWASP
AppSec DC
2009
Copyright © The OWASP Foundation
Permission is granted to copy, distribute and/or modify this document
under the terms of the OWASP License.
The OWASP Foundation
http://www.owasp.org
Amichai Shulman
 Background
 CTO and Co-founder at Imperva
 Application security pen-testing and consulting for
commercial and government organizations
 Security research for IDF
Leads the Application Defense Center (ADC)
Discovered over 20 commercial application
vulnerabilities
 Credited by Oracle, MS-SQL, IBM and others
Speaker at Industry Events
RSA, OWASP, Black Hat, Info Security UK, and
many others
Lecturer on Info Security
Technion - Israel Institute of Technology
OWASP
2
Ofer Shezaf
 Background:
 Design of Web Application Firewalls, at
Breach Security.
 Security research for the Israeli
Government.
 Open Source and Community projects:
 Officer, The Web Application Security
Consortium.
 Leader, OWASP Israeli chapter
 Project Lead WAFEC, The Web Application
Firewall Evaluation Criteria.
 Project Lead, WHID, The Web Hacking
Incident Database.
 Based out of Tel-Aviv, Israel.
OWASP
3
Agenda
The challenge of business logic bots
Business logic attacks
Business process automation:
 The friendly side of web automation
Business logic bots:
 Malicious web automaton
Solutions
Detection
Mitigation
OWASP
Business Logic Bots (BLBs)
A subset of Business Logic Threats (BLTs).
Compared to syntactic attacks:
Technical Attacks
Business Logic Attacks
Malformed requests
Normal requests
Invalid input values
Legitimate input values
Change functionality
Abuse functionality
Attack the application and
only indirectly the business
Attack directly the business
Usually a single request
Often multiple requests
OWASP
5
Business Logic Bots
What a pain in the neck…
Business Logic Threats
Normal requests
Legitimate input values
Abuse functionality
Attack directly the business
Often multiple requests
As a result
Hard to detect
Not always illegal
Key for mitigation!
OWASP
6
Web Automation
OWASP
7
Web Automation
The fact is that web automation is in wide use
Online form automation
Tracking competition
Personal and institutional stock trading
Indexing services
Comparative shopping
Web Services and other web APIs
Bottom line is that business level automation may or
may not be define as an attack based on the context
of things
Who is the source
Which part of the business logic is being invoked
OWASP
8
Born to be bad:
BUSINESS LOGIC BOTS
OWASP
Brute Force
Determine an unknown value by using an
automated process to try a large number
of possible values.
Can be used for:
Cracking login credentials
Guessing session identifiers
Guessing file and directory names (often
called
“Forceful Browsing”)
Credit card information such as CVV and
expiration date.
OWASP
10
Logic Bots Denial of Service
Locking resources:
Lock resources by doing nothing
Invoke and use slowly transaction based processes.
Abusing resource sensitive function:
Search
SQL Like injection
Regular expression injection.
Performed in a distributed fashion.
OWASP
Web Spam
Abuse public web pages to post
links in order to elevate site’s
ranking in search results (Abusive
SEO)
Advertise through Comment Spam
Abuse forms that send email for
spam
OWASP
Click Fraud
 Abuse pay per click advertising by generating automated
clicks.
 Performed by owners of web sites displaying ads,
competitors and vandals.
 Variants include:
 Referrer click fraud.
 CSRF click fraud.
OWASP
13
Hardcore Robotics
Queue Jumping
Ticketmaster confessed to “fighting like
the dickens“ queue jumping.
Travel agents known to automate air line
ticketing systems.
Auctions Sniping
Watching a timed online auction and
placing a winning bid at the last possible
moment giving the other bidders no time
to outbid the sniper.
Poll Skewing
OWASP
Gaming Bots
MUD, Virtual Worlds & Second Life
bots:
Gain Wealth, and turn it into money in
Second Life.
Scripted Clients
GUI Bots
Poker Bots:
Share information between several
bots at one table.
Monitor tables to choose the weak
ones.
Play well.
OWASP
Information Harvesting
Harvests:
E-mail and personal information
Competitive information
Record oriented information such as CVs
Entire Web sites for creating a mirror
Executed from:
Local computer
Distributed, potentially using bot net
Trojans, exploiting the victims credentials at the site
OWASP
16
ENOUGH WITH THE FUD!
OWASP
Solutions
The solution is comprised of two separate
problems
Detection
Mitigation
Detection
Detect automation (absolute)
Flag unauthorized use of automation (subjective)
Mitigation
Effective
Does not break application
OWASP
18
Detection – Basic Tools
Black listing:
IP Addresses (IP Reputation) – Anonymous Proxies,
TOR exit nodes, highly active bots
User Agents
Ad-hoc attack vector patterns
Ad-hoc comment spam patterns
Request structure
Missing / mismatch Host header
Irregular header combinations
Naïve, but eliminates the masses
OWASP
Detection – Proactive Techniques
Introduce extra content into the response
The extra content is interpreted in a different manner
by a human driven browser and by an automated tool
Must not affect visuals
Must not break application
Positive detection
Extra content affects a robot but not human
Examples:
 An invisible link outside of the view
 An invisible link generated by a script
Can be used to detect scraping
OWASP
20
Detection – Proactive Techniques
Negative Detection
Extra content affects a browser but not a robot
Examples
 Zero size imgae
 Reference created through script
 onError script of a missing image
Can be used to detect repetition attacks driven by
scripts like busy waiting, comment spam, etc.
OWASP
21
Detection – Frequency Measurement
Count the frequency of “events” within some
scope in a given time frame
Challenges
What’s an event?
What’s the best scope?
What’s the right threshold?
Allow detection of script related attacks and
brute force attacks
OWASP
22
Detection – Frequency Measurement
Flexible event definition
A combination of URL, parameter names and their values
Variance (all matches, different URLs only, same
parameter value only, etc.)
Session state (started, authenticated)
Examples
Access to /app/prod/
Access to /app/doAction.aspx with parameter cmd=commit
New session
Access to /app/prodDetails.aspx with different values of the
prodID parameter
 Access to /app/doPurchase.aspx with the same value in the
ccnumber parameter




OWASP
23
Detection – Frequency Measurement
Flexible Scope Definition
Some events must be performed in the context of a
session so it makes sense to measure the frequency
within a session
Some events need to be counted in the context of an
IP address (e.g. new sessions…)
Some attacks must occur in the context of a specific
application user
Detecting distributed attacks may require a wider
“system” conext
OWASP
24
Detection – Frequency Measurement
Threshold Definition
Some attacks are brutal enough and can be defined
in terms of absolute numbers
 Login brute force
 Exhaustive traversing of resources with “semi” predictable
identifiers
 Repetition attacks within a session context
Other attacks must be defined in terms of deviation
from either a short term (last minutes) or long term
(few days) observed frequency
 Distributed attacks
OWASP
25
Detection – Flow
Some attacks, either inherently or for
performance reasons bypass normal application
flow
Traversing a product catalog
Skipping transaction validation
Not easy to implement
Referer header can be forged
Flows are hard to define and track in modern
applications that use frames and AJAX
Can detect some types of forceful browsing and
man in the browser attacks
OWASP
26
Detection – Flow
Possible implementations
Track Referer (weak)
Introduce a nonce (may end up being very intrusive)
How to define / detect flows
Guided configuration
Learning algorithms
OWASP
27
Detection – Click Rate
Humans take time to respond (even the fast
ones)
Can be thought of as an extension to flow
Some observations:
Clickable events, within a session, need to be at some
minimal distance from one another
Within a session, over time, clickable events should
be relatively slow
Can detect general script attacks as well as man
in the browser attacks
OWASP
28
Detection - Summary
Will a single method do the trick?
I don’t think so.
Will there be false positives?
Yes!
Do I care?
No! Let me tell you why…
OWASP
29
Mitigation
Attacks are automated
I can’t prevent the attack from going on
I can however try to “defuse” its effects
Examples:





Slow down a brute force attack
Reduce the rate of a DDoS attack
Make the victim aware of a man-in-the-browser attack
Enforce flow on transactions
Disinformation
Preventative measures may increase the cost of
automation to the level that makes it much less
attractive for anything but high end targets
OWASP
30
Mitigation - Blocking
Dropping requests can only occur in very specific
cases
IP blacklists
User-agent blacklists
Strongly enforced flow (e.g. through nonce in a form)
Dropping requests that fail to answer the
challenges described in the following slides
OWASP
31
Mitigation – That Which Makes Us Human
Provide a Turing test that only a human can
solve.
Usually called CAPTCHA. Traditionally character
recognition
Other methods exists
Choose the correct description of an image
Solve a simple riddle
John had one thousand apples and five oranges. He ate
as many of his apples as there is letters in word
"apple". Also he ate two bananas :-). How many apples
John have?
OWASP
32
Mitigation – That Which Makes Us Human
There are automated tools and algorithms today
that solve CAPCHA’s of various types
I don’t care
If a brute force login program solves one CAPTCH per
second then it is ineffective
If a client solves a CAPTCHA faster than a human
being (no less than one second) then it can easily be
identified as a robot and further challenged (see next
slide)
OWASP
33
Mitigation – Throttling
Slowing down an attack is most often the best
way to make it ineffective.
A second of delay can make the difference for an
automated attack but will not be noticed by most
humans
Server side throttling may have sever impact on
server (quickly consume connection resources)
Client side computational challenges
Client is required to solve a computational challenge
that can be easily verified by server
Code for solving the challenge is introduced into the
response in the form of a script
OWASP
34
Mitigation – Throttling
Examples
Weak encryption of a random string
together with a public message
Reverse hash, partial hash collisions
Throttling can be per IP for those
requests that don’t have a session
yet and then applied by session to
those sessions that display
suspicious behavior.
OWASP
35
Mitigation – Adaptive Authentication
When automation is detected in the context of a
user (man in the browser)
Ask for additional authentication
Repeat password
Previously recorded questions
Makes the attack apparent to a victim
OWASP
36
Mitigation - Disinformation
Feed the client with bogus
information
A client follows a hidden link
Respond to the request with a page that
includes a large number of server
distinguishable random links
Whenever one of the random link is
request generate yet another random
page
A client that follows a hidden link
that was generated by a script
Respond with a page that include a
script that runs for a long time before
generating a new random link
OWASP
37
Mitigation - Summary
Mitigation methods should take into
consideration the possibility of false positives
Most often the system’s reaction to a suspected
automation attempt should not be blocking but
rather challenging the client
Legitimate clients are not materially affected
Automated clients become ineffective
OWASP
38
Summary
Automated business layer attacks are proliferating
today and expected to grow in number and
sophistication in the near term
Detecting and mitigating these attacks require a set
of sophisticated tools that are different than the
standard web application security tools
Some of the issues have nothing to do with the way
the application code is written
It’s bound to be a cat and mouse game as robots
become more sophisticated
As a consequence of the above, solutions should be
external to the application code
OWASP
39
Amichai Shulman, shulman@imperva.com
Ofer Shezaf, shezaf@xiom.com
OWASP
Web Spam example
Googling reveals many forums where
same bot left a message (as no
registration was required)
Links points to a semi
legitimate web site
A bot
generated
membership
request on
XIOM.COM
OWASP
Google
Trends
Time Most Influential People
Poll
Damage? Attack?
Poll Skewing
OWASP