The “Evil Bit” Revisited:
Blocking DDoS Attacks with
AS-Based Accountability
Dan Simon Sharad Agarwal Dave Maltz
Trustworthy Computing
April 8, 2006
1
The Solution to DoS is Already Here!
Network Working Group
Request for Comments: 3514
Category: Informational
S. Bellovin
AT&T Labs Research
1 April 2003
The Security Flag in the IPv4 Header
Firewalls ... and the like often have difficulty
distinguishing between packets that have malicious intent
and those that are merely unusual. The problem is that
making such determinations is hard. To solve this problem,
we define a security flag, known as the "evil" bit, in the
IPv4 header. Benign packets have this bit set to 0; those
that are used for an attack will have the bit set to 1.
Paraphrasing the rest of the RFC:
• Malicious applications MUST set the evil bit to 1
• Routers/firewalls SHOULD preferentially drop
pakcets with the evil bit set
2
Approaches to DoS on One Slide
• Ingress filtering
– Worthless until deployed everywhere, hence
undeployable
• Pushback filtering
– Requires PKI to authenticate requests
– Requires router hardware changes
– Reflection attacks render pushback useless
• Capabilities
– Capability server vulnerable to DoS
– Requires changing router HW and host SW
• Our solution: Leverage AS relationships and the evil bit
for incrementally-deployable ingress filtering and
pushback without router changes or a PKI
3
Outline
• The Problem of DoS
• Internet Accountability
• Achieving accountability among a club of
members with pair-wise trust
• Dealing with the world outside the club
• Determining if a member of the club has gone
bad
• Economics of Dos and Accountability
4
Understanding (D)DoS
• Basic structure: the attacker tries to get the
target to use up resources (memory, CPU,
bandwidth, etc.) dealing with DoS traffic
– Succeeds when the target’s remaining
resources are insufficient to handle everyone
else’s traffic
• The target’s only strategies are:
– Increasing resources enough to handle it all
– Distinguishing DoS traffic, and reducing the
resources expended on it
• E.g., identify sources of DoS traffic, and block
them
5
DoS and the Internet
webserver
AS2
access link
AS1
AS3
Company running a
website
You are here
Your customers are here
Some of them want to hurt you
6
application
(webserver)
Levels of DoS
AS2
‘bot detector
network stack
AS1
AS3
Can hit the application layer or the network layer
• App-layer DoS attack and defense are highly applicationdependent
– The attacker can exploit bugs or costly high-level
operations
– The defender can try to spot subtle distinguishing
cues in application-level traffic
7
application
(webserver)
Levels of DoS
AS2
‘bot detector
network stack
AS1
AS3
Network-layer DoS can attack any application
• Website serving a peak load of 100,000 hits/second,
@5KBytes/hit, needs a 4Gbps access link ($25$50K/month) ….
• ….Which is completely saturated by 4,000 1Mbps
broadband clients (about $400/week on the ‘botnet
market)
8
Levels of DoS
application
(webserver)
AS2
‘bot detector
network stack
AS1
AS3
Observations:
• If new attackers appear faster than they can be classified,
then all is lost (IP spoofing/DHCP exacerbates)
• Need to shed all load from attacker once identified
• ASes have long-lived relationships with each other
• ASes generally have non-transient relationships with their
customers (e.g., billing information)
9
Sidebar: Isn’t End-Host Security
the “Real” Problem?
• True, compromised end hosts (“botnets”) are a major
source of unwanted traffic
• But….suppose end hosts were bullet-proof
– E.g., you could run “SETI@home” completely safely
• Introducing “DDoS@home”....
– Does something nice for the user (use your
imagination)
– Borrows spare cycles and bandwidth in return, with
which to launch DDoS attacks
– The user may or may not even know or care what it
does, because….
– …The user’s computer and data are completely safe
• Conclusion: End hosts aren’t the whole problem
– The solution needs to involve the network, as well
10
Outline
• The Problem of DoS
• Internet Accountability
• Achieving accountability among a club of
members with pair-wise trust
• Dealing with the world outside the club
• Determining if a member of the club has gone
bad
• Economics of Dos and Accountability
11
What Is Accountability?
Two components:
• Identification: Receivers of traffic can distinguish it by
source
– Must be based on some persistent attribute
• I.e., difficult/expensive for the originator to change
• Think “IP address”—but something more durable is needed
• Defensibility: Receivers can choose to avoid (block) traffic
from a source with a particular persistent attribute
– Requires identification, but doesn’t automatically follow
from it
Allows a DoS target to distinguish DoS traffic sources (not
just IP address), and block all traffic from them
12
Outline
• The Problem of DoS
• Internet Accountability
• Achieving accountability among a club of
members with pair-wise trust
• Dealing with the world outside the club
• Determining if a member of the club has gone
bad
• Economics of Dos and Accountability
13
Implementing Accountability on the Internet
(Among a Club with Pair-wise Trust)
E1
Home
Router
`
E2
AS1
BRAS
AS2
`
CRM
E3
`
W1
FRS-1
CRM
FRS-2
`
`
`
• “Total” Ingress Filtering
– Every packet is required to have an honest source address
– Covers all traffic (e.g., DNS spoofing eliminated!)
• Filtering on request by source-destination pair
– Requestor identifies “source” by (IP address, port, time)
– Too many filters against a source leads to other measures
• Rate-limiting, offers for support upgrades, warnings, charging
14
Implementing Accountability on the Internet
(Among a Club with Pair-wise Trust)
E1
Home
Router
`
E2
AS1
BRAS
AS2
`
CRM
E3
`
W1
FRS-1
CRM
FRS-2
`
`
`
Both measures are best implemented at the source ISP
• Can identify, distinguish and filter an individual offending
machine/user at the ingress point (even assuming DHCP, NAT, etc.)
• Downstream ASes are spared the traffic
• Multiple paths aren’t a problem
• Habitual offenders are more easily recognized
15
Secure Relay of Filtering Requests
• Assume for now:
– Peer ISPs cooperate—including ingress filtering
• Basic Design:
– Every AS/ISP has a Filter Request Server (FRS)
– Requestor sends a request to its local ISP’s FRS
– FRS forwards the request along a chain of FRSs to
the source’s ISP: “filter all traffic from source to
requestor”
• Request does not need to follow the same path as the
offending traffic
– Source’s ISP identifies the source and applies the
filter
16
How FRSs Work
E1
`
E2
Home
Router
AS2
AS1
R2
FRS-2
R4
AS4
DSLAM
R1
`
W1
R6
E3
R3
AS3
CRM FRS-1
`
E4
R5
FRS-4
FRS-3
`
17
Attacks on the FRS System
• Customer asks FRS to filter traffic to another customer
– Can’t happen – FRS only allows filters on traffic to
requestor
– Ingress filtering prevents spoofing and crypto is easy
• An ISP injects spoofed Filter Requests
– Means an ISP is breaking its peering agreements
• Presumably a rare, serious event
– Heavy-duty investigation using sampling, logs,
cooperation
• DoS on FRSs
– FRS can filter traffic to itself!
– ISP can charge for right to send filter requests, and
scale FRS accordingly
18
Attacks on the FRS System
• Framing attacks (false, non-spoofed complaints)
– Only relevant when they accumulate, hurting a
source’s reputation
• A single framer can block anyone he/she wants (and who
cares?)
– Solution: delay punitive measures to allow resolution
• Gives the customer a chance to complain of framing
• Overwhelm a router’s ability to filter a customer
– Apply many useless filters against a bot, so real
filters don’t take
– Solution: evict the customer from the club
19
Outline
• The Problem of DoS
• Internet Accountability
• Achieving accountability among a club of
members with pair-wise trust
• Dealing with the world outside the club
• Determining if a member of the club has gone
bad
• Economics of Dos and Accountability
20
Incremental Deployment and the “Evil Bit”
AS2
Unaccountable
AS
AS5
Unaccountable
AS
AS1
AS3
AS4
• Packets from “unaccountable” peers get “evil bit” set
– Marking occurs at ingress to accountable AS
– Bit stays set as the packet travels through
accountable ASes
– Behavior enforced by peering agreement
21
Incremental Deployment and the “Evil Bit”
AS2
Unaccountable
AS
AS5
Unaccountable
AS
AS1
AS3
AS4
• ISPs can de-prioritize “evil” traffic sent to their customers
– On request, or when traffic to the customer is too heavy
– DiffServ mechanisms are already available on most routers
– Incentive to other ISPs to become accountable
• ISPs can also slap an “evil bit” on repeat offenders’ traffic
– Cheaper than filtering out a large list of destination addresses
– Incentive for the offender to stop attacks/remove malware
22
Reflection Attacks
• Attacker “redirects” attack packets through an innocent “reflector”
– E.g., TCP SYN with (forged) source address of the attack
target—results in an ACK sent to the target
– Doesn’t magnify the attack, but helps disguise its source attack
packets can shed their “evil bit”
• Reflectors are fundamental to IP architecture
– IP depends on hosts/routers replying to single packets
– Path MTU Discovery, TCP RST, ICMP TTL Exceeded
– Routers, hosts – all can be used as reflectors
• Only two choices
– Hosts/routers reply only to packets from authenticated sources –
breaks all connectivity to hosts using different or no
authentication
– Host/router software must change to reflect some characteristic
of incoming packet in reply
23
Reflection Attacks - Solution
• Hosts need to preserve the “evil bit” setting of
incoming packets when replying to them
– Requires changes to host network stacks
• But these are necessary in any event, to protect
against reflection attacks—regardless of the DoS
defense scheme
• Incremental deployment possible
– Accountable ISPs probe customers for
reflection
– Set “evil bit” on hosts not reflecting evil bits
properly
24
Outline
• The Problem of DoS
• Internet Accountability
• Achieving accountability among a club of
members with pair-wise trust
• Dealing with the world outside the club
• Determining if a member of the club has gone
bad
• Economics of Dos and Accountability
25
What Motivates an AS to Follow the
Accountablity Club Rules?
• Active malfeasance among ASes can be
expected to be rare
– Peering agreements are contracts = lawyers
• So long as misbehavior is observable to the
world, peer pressure keeps ASes in line
– Detection must be possible
– Need not be automated
– Failure to behave means peer ASes will set
evil bit on the offender’s traffic
• BGP is a positive example
26
Detecting Fraudulent “Accountable” ASes
AS2
D
Unaccountable
AS6
Unaccountable
AS7
AS5
AS1
AS3
AS4
S1
Easiest case to detect: AS claims to install filters when
requested, but doesn’t
• Request forwarded and FRS ACK received, but packets
still arrive and packet have evil bit clear
• Each AS tasks the upstream AS to determine why the
filter wasn’t applied, on pain of having evil bit set “getting
tossed out of the accountable club”
27
Detecting Fraudulent “Accountable” ASes
AS2
D
Unaccountable
AS6
Unaccountable
AS7
AS5
AS1
AS3
AS4
S1
S1
Hardest case to detect: AS claims to perform ingress
filtering, but doesn’t
• Packets with evil bit clear arrive at a host
• Packets don’t stop when host requests filtering
• Collaboration among neighbor accountable ASes shows
packets not entering the fraudulent AS
28
Outline
• The Problem of DoS
• Internet Accountability
• Achieving accountability among a club of
members with pair-wise trust
• Dealing with the world outside the club
• Determining if a member of the club has gone
bad
• Economics of Dos and Accountability
29
Economic Justification
Say you want to protect against 50,000 bots@128Kbps/bot....
• Roll-your-own solution:
– 6.4 Gbps bandwidth (~10 OC-12s at $30K/month) + enough
magic scrubber boxes ($110K/OC-12) = ~$10M over 3 years
• Scrubbing services provided by ISP:
– $210K/month for 3 OC-48s’ worth of scrubbing = ~$7.5M over 3
years
• Deploying accountability:
– Hard to say how much this would cost
– But we have data on the cost of two huge communication
infrastructure overhauls: number block pooling and local
wireless number portability
– If accountability is comparable, it would cost about $60-230M for
the whole Internet
• Bottom line: Somewhere between 6 and 30 large orgs could
probably finance deployment of Internet accountability by
themselves, out of their 3-year savings estimates
30
Deployment Strategies
• Three primary types of ISPs
– Serve end-users: Verizon, Comcast, ...
– Serve servers and enterprises: SAVIS, AboveNet, parts
of ATT & Sprint, ...
– Serve other networks (transit): UUNET, ATT, Sprint, ...
• Deployment realities
– Deployment costs highest for ISPs serving end-users
– Revenue potential greatest for ISPs serving servers
– But these networks often peer directly! Now money
can flow and mutual benefit arise...
• Who deploys first? Pairs of Server ISPs and User ISPs
31
Deployment Over Time
• DoS on the Internet slowly goes away
– A host with too many filters has evil bit set
– ‘botnets become harder to build and aren’t
useful for as long
– Ultimately, cost for links to DoS a server
become greater than the cost of links to connect
the server
• Deploying accountability in transit network is
cheap
– Just need an FRS to do pass-thru
– Probably not early adopters though
32
Related Work
• Identification
– TCP handshake for identification
• No good for UDP, DNS, TCP SYN floods, transient IP addresses
– End-to-end (IPSec, HIP)
• Requires PKI, DNSSec or some other global identity infrastructure
– Traceback/packet-marking
• Partial, statistical information, doesn’t help against transient IP
addresses
• Highly vulnerable to reflection attacks
• Defensibility
– End-host filtering (possibly with packet-marking)
• End hosts still have to have enough bandwidth to handle attacks
– Capabilities
• Don’t protect the capability issuer, and require state in the network
to prevent capability abuse
– “Active filtering”
• Packet-marking + filtering scattered across the network (not just at
source)
33
Conclusions
• Network-level DDoS is a big problem on the
Internet today
– It’s costing a lot of people a lot of money
– Solving it will require costly changes
• Accountability actually solves the problem
– DDoS traffic can be identified and blocked at
the source
• Accountability is actually deployable
– Ingress and by-destination on-request filtering
are technically straightforward to deploy
– The “evil bit” enables incremental deployment
• Accountability is cheaper than living with DDoS
34
Questions?
Thanks!
35
Our Solution in 1 Slide
• Leverage the persistence of relationships among
ASes and their customers to create a working
“pushback” packet filtering system
• Use the “evil” bit to deal with non-participating
ASes, achieve incremental deployability, and
attain scalability
• Provide techniques to discover and cope with
ASes that claim to be good citizens, but actually
cheat or are lazy
• Show that our infrastructure is cheaper than
alternatives
36
Sidebar 2: Computer Networking People,
Please Cover Your Ears…
• There exists a large-scale, successful network
today that doesn’t have a DoS problem….
• …Despite the fact that DoS attacks are
breathtakingly easy on it
• The reason: Accountability
– Targets know who’s attacking, and can ask
for the network to block all traffic from them
(Okay, Computer Networking folks—you can
uncover your ears now….)
37
Attacks
• Perhaps state that our scheme is targeted at end
hosts (clients, servers). If a router is
compromised, forget flooding style attacks,
you've got more serious BGP attacks that can
get rid of your legitimate traffic.
38
Deployment
• Pairings of ISPs specializing in hosting servers and ISPs
specializing in serving home users
• Two tiers of filter requestors
– Big servers are stable, and pay for the right to request
lots of filters (they’re the DoS targets, after all)
– Small clients rarely if ever need to request filters
• ASes/ISPs can include “accountability” in their peering
agreements
– Mutually agree to deploy ingress filtering and accept
filtering requests from destinations (really easy for
core ASes without end-host clients)
39
The Evil Bit In Action
AS2
AS5
AS1
AS3
AS4
40