Paper Reading:
Reporter: Shao-Yu Peng(彭少瑜)
Date: 2013/10/28
Outline
•
•
•
•
•
•
•
•
Purpose
Introduction
Fluxing features of botnets
Features detection techniques
Comparison and evaluation
Fluxing mitigation
Future work
Conclusion
2
/33
Purpose
• Summarized and classified the latest botnet fluxing features
and detection techniques.
• Compared and Evaluated the surveyed techniques against
multiple criteria.
3
/33
Introduction
• Botnet: A group of computers(bots/zombies) which controlled
by the botmaster.
• In recent years, fluxing techniques have been applied to evade
detection.
4
/33
Fluxing Features of botnet
5
/33
Fluxing features of botnets
• Fluxing methods are used to evade detected by hiding the
domain-IP mappings.
• In our survey, we focus on two advanced mechanisms:
1. Fast flux(FF):
a set of IP addresses->
a unique domain name
2. Domain flux(DF):
a set of domain names->
a unique IP address
6
/33
Fast Fluxing, RRDNS and CDNs
• Ways to distribute loads of online services:
1. RRDNS(Round-robin DNS):
Round-robin to response DNS requests.
2. CDNs(Content Distribution Networks):
Computes the nearest servers to response.
3. Fast fluxing:
Same idea but change entries more rapidly.
7
/33
Measuring and Detecting Fast-Flux Service Network
Thorsten Holz
Fast Fluxing Network
• Characters:
Short TTLs, share one large IP pools…etc.
• Categories:
1. Single flux
2. Double flux
8
/33
Fast Fluxing Network
9
/33
http://www.honeynet.org/files/images/web-diagram.gif
https://job.honeynet.org/files/images/dns-diagram.gif
Domain Fluxing Network
• Server and bots generates domain names through same
algorithm(consistently).
• Example:
Torpig
10
/33
Torpig:Bot
Domain generation
algorithm
Current week, year
Domain name 1
success
Domain generation
algorithm
master
failed
Domain name 2
Current day
success
failed
Configuration file
Hard-coded
domain names
11
/33
Features detection techniques
Fast fluxing
12
/33
Detection techniques
Measuring and Detecting Fast-Flux Service Networks
FF detection 1:
• Holz et al.:
• Distinguish btw normal network and fast fluxing network, and
• score a networks by:
1. #of IP-domain mappings in all DNS lookups,
(more->higher prob. to be botnet)
2. #of nameserver records in one domain lookup,
(more->higher prob. to be botnet)
3. #of autonomous system in all IP-domain pairs
(more->higher prob. to be botnet)
• Limitation on detecting FFSN(benign) & FFAN(malicious)
13
/33
Detection techniques
Collaborative Detection of Fast-Flux Phishing Domains
FF detection 2:
• Zhou et al.:
1. To speed up Holz method
2. Improvement speed by combining results:
(1) From different DNS servers;
Build and share one suspicious IP address list.
(2) From different suspect FF domains.
Compare responses from domains to speed up confirmation.
14
/33
(1)
Switch Address blacklist
Server 1
Server 3
Switch Address blacklist
Switch Address blacklist
Server 2
Each server: List’ = List 1 ∪ List2 ∪List3
(2)
Response 1
Unknown
domain
Response 4
Server
FF domain 1
Response 2
FF domain 2
Response 3
FF domain 3
List’= Response 1∪ Response 2 ∪ Response 3
15
/33
Detection techniques
Real-time detection of as flux service networks
FF detection 3:
• Caglayan et al.:
1. Monitor the DNS of a website by minutes.
2. Sensors, FF monitor/database, FFM classifier
3. Sensors monitor parameters including TTL…etc.
and store into database.
4. Classifier evaluate a website with the analytic
data in database.
16
/33
FF domain
FF domain
FF monitors
FFM database
Sensor
Classifier
Unknown domain
Unknown Website
with rapidly changed IP
17
/33
Detection techniques
Detecting malicious flux service networks through
passive analysis of recursive DNS traces
FF detection 4:
• Perdisci et al.:
• Detect malicious ones from FFSN.
1. Monitoring FFSN traffic with a pre-filter by four features:
(1) Short TTL,
(2) The change rate of the set of resolved IPs returned ,
(3) A large number of resolved IPs,
(4) Resolved IPs scattered across different networks.
2. Clustered domains with high relations
3. Classified domains according to the resolved IP address
4. Build a network classifier based on above data.
FFSN=Fast-flux service network
FFAN=Fast-flux attack network
18
/33
Detection techniques
Fast-flux attack network identification based on agent lifespan
FF detection 5:
• Yu et al.
• Distinguish FFSN and FFAN by agent lifespan.
1. Send request once per hour during 24 hours.
2. FFSN: 24/7 available; FFAN: unpredictable.
3. AOR(average online rate/24 hours)
4. MAR(minimum available rate/history record)
5. Detector judges btw FFAN and FFSN by AOR
and MAR record by monitors.
19
/33
Features detection techniques
Domain fluxing
20
/33
Detection techniques
Your botnet is my botnet: analysis of a botnet takeover
DF detection 1:
• Stone-Gross et al.:
1. To determine the size of a botnet
2. Research on real world botnet –Torpig
3. Register the .com and .net domain which would
be used by the botnet.
4. Log requests and record network traffic.
5. Determine the size by counting unique nodes.
21
/33
Detection techniques
Beyond blacklists: learning to detect malicious web sites from suspicious URLs
DF detection 2:
• Ma et al.:
• Distinguish domain fluxing network and normal network.
1. URL analysis based.
2. Lexical features and host-based features
(1) Lexical:
URL length, #of dots in URL, bag-of-words…etc.
(2) Host-based:
IP, domain name, location, connection speed…
3. Independent of content and structure.
4. Combination of all features -> highest accuracy.
22
/33
Detection techniques
Identifying suspicious activities through DNS failure graph analysis
DF detection 3:
• Jiang et al.:
• Distinguish domain fluxing network and normal network, and
classified.
1. Failed DNS queries come mainly from malicious activities.
2. DNS failure graph
(bots with same DGA will create dense failure graph)
4. Analyze the graph structure and refer to domain name
blacklists.
23
/33
Detection techniques
Phishnet: Predictive blacklisting to detect phishing attacks
DF detection 4:
• Prakash et al.:
• Evaluation based on blacklists.
• Since Black listing method needed to exactly match URL, it is easy to evade.
• Model: Score new URL against an existing blacklist with 5 heuristics:
1. Replace the top-level domains
(4)
(3) ex:
2. IP address equivalence
www.abc.com/online/singin/ebay?XYZ
www.abc.com/online/singin/ebay.htm
(Same IP->change dir/path)
www.xyz.com/online/singin/paypal?ABC
www.xyz.com/online/singin/paypal.htm
Change query->
filename->
3. Directory structure similarity
www.abc.com/online/singin/ebay?ABC
www.abc.com/online/singin/paypal.htm
(different IP, similar path-> change filename)
www.xyz.com/online/singin/paypal?XYZ
www.xyz.com/online/singin/ebay.htm
4. Query string substitution
(5) ex:
(Same structure->change query)
www.abc.com/online/singin/ebay .htm
5. brand name equivalence
Change brand name->
www.abc.com/online/singng/yahoo.htm
24
/33
Detection techniques
Detecting algorithmically generated malicious domain names
DF detection 5:
• Yadav et al.
• Distinguish DF domain names from normal domain names.
1. Identify domain names generated by
algorithm by spelling or pronounceable features.
2. Group DNS queries by TLD/IP-address
3. For each group, use Jaccard index to
characterize alphanumeric distribution.
25
/33
Suspicious URL, ex:
ickoxjsov.botnet.com
Break into bigrams
Database of
non-malicious bigrams
Ic,ck,ko,ox,xj,js,so,ov
Subset with 75% of bigrams
ex:
the quick brown fox jump sover the lazy dog
Average JI
Calculate
JI = (A∩B)/(A∪B)
ex:
6/(8+35-6) = 0.16
26
/33
Comparison between techniques
27
/33
Comparison between techniques
• DF:
• FF:
• 4 criteria:
• 5 criteria:
1. Accuracy
1. Real-time
2. Speed
2. Accuracy
3. Distinguish FFSN VS. FFAN 3. Passive or active
4. Mining based
4. Speed
5. Mining based
Above these criteria,
Is this meaningful to compare the algorithms with different goals?
28
/33
A Survey on Latest Botnet Attack and Defend
dash line: not discussed or unclear in a paper
29
/33
Fluxing Mitigation
• Need collaboration of both registers and ISPs.
• Blacklisting-related method is almost the only
way.
30
/33
Future directions
• Data mining can be used widely to extract features.
• Graph spectra can be employed to study botnets.
• How to get the trust of remote owners which has
compromised computers.
• Predict botnet writers new developed strategies.
31
/33
Conclusion
• Advantages:
Survey on latest fluxing detection techniques of botnet.
• Drawbacks:
The meaning of comparison btw algorithms with different
purposes is vague.
32
/33
Thank you for listening