First Step Towards Automatic Correction
of Firewall Policy Faults
Fei Chen
Alex X. Liu
Computer Science and Engineering
Michigan State University
JeeHyun Hwang Tao Xie
Computer Science
North Carolina State University
What do we do here?
 Most firewall policies are poorly configured and contain faults.
[Wool 2004 & 2010]
─ A coworker may mess up your firewall rules
─ Any modification may introduce firewall faults.
 We invent methods for fixing firewall policies automatically.
─ We first model 5 types of faults.
─ For each type of faults, we develop an algorithm to fix them.
─ Given a faulty firewall policy, we propose a systematic method to fix the faults
automatically using the 5 algorithms.
2/29
Roadmap
 Background
─ Firewalls
─ Firewall Policies
─ Firewall Policy Faults
 Technical Challenges
 Fault model of firewall policies
─ Five types of faults
 Problem formalization
 Our solution
 Experimental results
3/29
Background – Firewalls
 A firewall checks all outgoing and incoming packets
 The firewall policy decides whether to accept or discard a packet
Private Network
Outgoing Packets
Incoming Packets
Internet
Firewall
4/29
Background – Firewall Policies
 A firewall policy is usually specified as a sequence of rules
 Each rule consists of a predicate and a decision.
─ A predicate typically includes five fields:
source IP, destination IP, source port, destination port, protocol type
─ Typical decisions are accept and discard.
Firewall Policy
Src IP
Dst IP
Src Port
Dst Port
Protocol
Decision
r1
1.2.3.*
192.168.1.1
*
25
TCP
Accept
r2
1.2.3.9
192.168.1.1
*
25
*
Discard
r3
*
*
*
*
*
Discard
Src IP
Dst IP
Src Port
Dst Port
Protocol
Payload
1.2.3.5
192.168.1.1
78
25
TCP
Packet
 Conflict Resolution: first-match
5/29
Background – Firewall Policy Faults
 Most firewall policies are poorly configured and contain faults.
[Wool 2004 & 2010]
 It is dangerous to have faults in a firewall policy. A policy fault
─ either allows malicious traffic to sneak into the private network
─ or blocks legitimate traffic and disrupts normal business processes
 A faulty policy evaluates some packets to unexpected decisions.
─ Such packets are called misclassified packets of a faulty firewall policy
 Manually locating and correcting firewall faults are impractical.
─ A firewall may consist of thousands of rules
 Automatically correcting firewall faults is an important problem.
6/29
Roadmap
 Background
─ Firewalls
─ Firewall Policies
─ Firewall Policy Faults
 Technical Challenges
 Fault model of firewall policies
─ Five types of faults
 Problem formalization
 Our solution
 Experimental results
7/29
Three Key Technical Challenges
 It is difficult to determine the number of policy faults and the
type of each fault.
─ A set of misclassified packets can be caused by different types of faults
and different number of faults.
 It is difficult to correct a firewall fault.
─ A firewall policy may consists of a large number of rules.
─ Each rule has a predicate over multi-dimensional fields.
 It is difficult to correct a fault without introducing other faults
─ Due to the first match, correcting faults in a firewall rule affects the
functionally of all the subsequent rules.
8/29
Roadmap
 Background
─ Firewalls
─ Firewall Policies
─ Firewall Policy Faults
 Technical Challenges
 Fault model of firewall policies
─ Five types of faults
 Problem formalization
 Our solution
 Experimental results
9/29
Fault Model of Firewall Policies (1/2)
 We propose a fault model that includes five types of faults
(1) Wrong order: the order of firewall rules is wrong.
Src IP
Dst IP
Src Port
Dst Port
Protocol
Decision
r1
1.2.3.*
192.168.1.1
*
25
TCP
Accept
r2
1.2.3.9
192.168.1.1
*
25
*
Discard
Correction technique: Order Fixing
(2) Missing rules: some rules are missed in the firewall policy.
r*
Src IP
Dst IP
Src Port
Dst Port
Protocol
Decision
r1
1.2.3.*
192.168.1.1
*
25
TCP
Accept
r2
1.2.3.9
192.168.1.1
*
25
*
Discard
Correction technique: Rule Addition
(3) Wrong predicates: the predicates of some rules are wrong.
r1
Src IP
Dst IP
Src Port
Dst Port
Protocol
Decision
1.2.3.*
192.168.1.1
*
25
TCP
Accept
Correction technique: Predicate Fixing
10/29
Fault Model of Firewall Policies (2/2)
(4) Wrong decisions: the decisions of some rules are wrong.
Src IP
Dst IP
Src Port
Dst Port
Protocol
Decision
r1
1.2.3.*
192.168.1.1
*
25
TCP
Accept
r2
1.2.3.9
192.168.1.1
*
25
*
Discard
Correction technique: Decision Fixing
(5) Wrong extra rules: some rules are not needed in the policy.
Src IP
Dst IP
Src Port
Dst Port
Protocol
Decision
r1
1.2.3.*
192.168.1.1
*
25
TCP
Accept
r2
1.2.3.9
192.168.1.1
*
25
*
Discard
r3
*
*
*
*
*
Discard
Correction technique: Rule Deletion
Each operation of these five techniques is called a modification.
11/29
Roadmap
 Background
─ Firewalls
─ Firewall Policies
─ Firewall Policy Faults
 Technical Challenges
 Fault model of firewall policies
─ Five types of faults
 Problem formalization
 Our solution
 Experimental results
12/29
Detection of Faulty Firewall Policies
 A faulty firewall policy is detected when
─ administrators find that the policy allows some malicious packets or blocks
some legitimate packets.
Faulty Firewall
Policy
×
Malicious
Packets
Legitimate
Packets
Administrator
 These packets cannot provide enough information about the faults
─ The number of these observed packets is typically small
 Bruteforce testing every possible packets needs 2104
 How to generate test packets for faulty firewall policies?
13/29
Generating Test Packets for Faulty Policies
 We employ the automated packet generation techniques in [Hwang
et al. 2008] to generate test packets
 Administrators identify passed/failed tests automatically or manually
According to security requirements for the firewall policy,
─ If the decision of a packet is correct, administrators classify it as a passed test.
─ Otherwise, administrators classify it as a failed test.
Faulty Firewall
Policy
Packet
Generation
Classify
Packets
Passed
Packets
Failed
Packets
14/29
Problem Statement
 Input:
(1) A faulty firewall policy FW
(2) A set of passed tests PT, |PT|≥0
(3) A set of failed tests FT, |FT|>0
 Output:
A sequence of modifications <M1, …, Mm>, where Mj (1≤j ≤m)
denotes one modifition, satisfies the following two conditions:
(1) After applying <M1, …, Mm> to FW, all tests in PT and FT
become passed tests.
(2) No other sequence that satisfies the first condition has the
smaller number of modifications than m.
 This is a global optimization problem and hard to solve because
─ a policy may consist of a large number of rules, and
─ different combinations of modifications can be made.
15/29
Roadmap
 Background
─ Firewalls
─ Firewall Policies
─ Firewall Policy Faults
 Technical Challenges
 Fault model of firewall policies
─ Five types of faults
 Problem formalization
 Our solution
 Experimental results
16/29
Automatic Correction of Firewall Policy Faults
 We propose a greedy algorithm to address this problem.
─ For each step, we correct one fault in the policy such that |PT| increases.
─ To determine which technique should be used, we try the five correction
techniques and then find the one that maximizes |PT|.
Faulty Firewall
Policy
Order
Fixing
Passed
Packets
Rule
Addition
Predicate
Fixing
Decision
Fixing
Failed
Packets
Rule
Deletion
No
|Failed Tests|=0 ?
Yes
Fixed Firewall Policy
17/29
Running Example
r1: F1  [1, 5]  F2  [1, 10]  a
r2: F1  [1, 6]  F2  [3, 10]  a
r3: F1  [6,10]  F2  [1, 3]  d
r4: F1  [7,10]  F2  [4, 8]  a
r5: F1  [1,10]  F2  [1, 10]  d
A faulty firewall policy
p1: (3, 2)  a
p2: (5, 7)  a
p3: (6, 7)  a
p4: (7, 2)  d
p5: (8,10) d
A set of passed tests
p6: (6, 3)  d
p7: (7, 9)  a
p8: (8, 5)  d
A set of failed tests
18/29
Order Fixing (1/2)
 Swapping every two rules is computationally expensive.
─ There are (n-1)(n-2)/2 pairs of rules that can be swapped
 We use all-match firewall decision diagrams (all-match FDDs)
[Liu et al. 2008] as the core data structure.
─ Any firewall policy can be converted to an equivalent all-match FDD.
r1: F1  [1, 5]  F2  [1, 10]  a
r2: F1  [1, 6]  F2  [3, 10]  a
r3: F1  [6,10]  F2  [1, 3]  d
r4: F1  [7,10]  F2  [4, 8]  a
r5: F1  [1,10]  F2  [1, 10]  d
F1
[1, 5]
[6, 6]
F2
[1, 2]
1,5
[7, 10]
F2
[3, 10] [1,2]
1,2,5
3,5
F2
[4,10]
[3,3]
2,3,5
2,5
[1,3]
[4,8]
3,5
4,5
[9,10]
5
19/29
Order Fixing (2/2)
 All-match FDD has the following nice property.
Swapping two rules is equivalent to swapping the sequence
numbers of the two rules in the terminal nodes of all-match FDD
<r1, r2, r3, r4, r5>  <r1, r3, r2, r4, r5>
F1
[1, 5]
[6, 6]
F2
[1, 2]
1,5
[7, 10]
F2
[3, 10] [1,2]
1,2,5
3,5
F2
[4,10]
[3,3]
3,2,5
2,3,5
2,5
[1,3]
[4,8]
3,5
4,5
[9,10]
5
 For the running example, this technique can find that swapping
r2 and r3 can increase |PT| by 1
─ change the failed test (6, 3)  d to a passed test
20/29
Rule Addition
 Bruteforce addition for each position is computationally expensive
─ The number of possible rules that can be added for each position is O(2204).
 The basic idea of rule addition is that for each position
─ Find all possible failed tests that can be corrected by adding a rule
r*: F1  [ , ]  F2  [ , ]  dec
r1: F1  [1, 5]  F2  [1, 10]  a
r*: F1  [ , ]  F2  [ , ]  dec
r2: F1  [1, 6]  F2  [3, 10]  a
r*: F1  [ , ]  F2  [ , ]  dec
r3: F1  [6,10]  F2  [1, 3]  d
r*: F1  [ , ]  F2  [ , ]  dec
r4: F1  [7,10]  F2  [4, 8]  a
r*: F1  [ , ]  F2  [ , ]  dec
r5: F1  [1,10]  F2  [1, 10]  d
p7: (7, 9)  a p6: (6, 3)  d p8: (8, 5)  d
p7: (7, 9)  a p6: (6, 3)  d p8: (8, 5)  d
p6: (6, 3)  d
p7: (7, 9)  a
p8: (8, 5)  d
p7: (7, 9)  a
p8: (8, 5)  d
p8: (8, 5)  d
p7: (7, 9)  a
─ Compute a rule that matches the maximum number of failed tests
● For adding a rule between r1, r2, we can compute F1  [6, 8]  F2  [3, 5]  d to
correct two failed tests p6: (6, 3)  d and p8: (8, 5)  d .
21/29
Evaluation Setup
 We generate faulty firewall policies from 40 real-life policies.
─ Each faulty policy contains one type of fault, and the number of faults
ranges from 1 to 5.
─ For each faulty policy, we employed the packet generating technique
[Hwang et al. 2008] and then classified them into passed and failed tests
─ We applied our greedy algorithm to produce the fixed policy.
 Methodology
─ Difference ratio over FWreal, FWfaulty, and FWfixed
Real Policy
FWreal
Faulty Policy
FWfaulty
Fixed Policy
FWfixed
(FWreal , FWfixed )
(FWreal , FWfaulty )
─ The average number of modifications
22/29
Roadmap
 Background
─ Firewalls
─ Firewall Policies
─ Firewall Policy Faults
 Technical Challenges
 Fault model of firewall policies
─ Five types of faults
 Problem formalization
 Our solution
 Experimental results
23/29
Effectiveness (1/4)
 For wrong decision faults
The percentages of fixed policies that are equivalent to their corresponding reallife policies are 73.5%, 68.8%, 63.7%, 59.3%, and 53.8%, respectively.
24/29
Effectiveness (2/4)
 For wrong order faults
The percentages of fixed policies that are equivalent to their corresponding reallife policies are 69.7%, 64.2%, 59.7%, 54.3%, and 48.9%, respectively.
25/29
Effectiveness (3/4)
 For wrong extra rule faults
The percentages of fixed policies that are equivalent to their corresponding reallife policies are 68.3%, 63.5%, 59.3%, 53.2%, and 47.3%, respectively.
26/29
Effectiveness (4/4)
 In terms the number of modifications
The number of modifications of our approach is close to the minimum number.
27/29
Contributions
 Propose the first comprehensive fault model for firewall policies
 Propose the first systematic approach that can automatically
correct all or part of the misclassified packets of a faulty policy.
 Conduct extensive experiments on real-life firewall policies to
evaluate the effectiveness of our approach.
28/29
Questions
Thank you!
29/29
Reference
 [Wool 2004] A quantitative study of firewall configuration
errors. IEEE Computer 37, 6 (2004), pp. 62–67.
 [Wool 2010] Trends in Firewall Configuration Errors:
Measuring the Holes in Swiss Cheese. IEEE Internet Computing
14, 4 (2010), pp. 58–65 .
 [Hwang et al. 2008] Systematic structural testing of firewall
policies. In Proceedings of IEEE International Symposium on
Reliable Distributed Systems (SRDS) (2008), pp. 105–114.
 [Liu et al. 2008] All-match based complete redundancy removal
for packet classifiers in TCAMs. In Proceedings of IEEE
Conference on Computer Communications (INFOCOM) (2008),
pp. 574–582.
30/29
Limitation of Prior Art
 A firewall fault localization scheme was proposed [Marmorstein
et al. 2007].
─ Find failed tests that violate the security requirement of a firewall policy
─ Use the failed tests to locate two or three faulty rules in the policy.
 Drawbacks
─ Many types of faults cannot be located.
For example, wrong order of firewall rules cannot be located.
─ Even if a faulty rule is located, it may not be corrected by just changing
the faulty rules.
For example, if a firewall policy misses one rule, change existing rules
cannot correct the fault.
─ The scheme only were applied to a simple policy with 5 rules, which
cannot strongly demonstrate the effectiveness of the scheme.
31/29
Predicate Fixing
 The basic idea of predicate fixing is similar to rule addition.
 However, there are two major differences.
─ For fixing ri’s predicate, compute only a rule with the same decision of ri.
─ After fixing ri’s predicate, the original rule ri does not exist.
r1: F1  [1, 5]  F2  [1, 10]  a
r2: F1  [1, 6]  F2  [3, 10]  a
r3: F1  [6,10]  F2  [1, 3]  d
r4: F1  [7,10]  F2  [4, 8]  a
PT(i+1)a
p3
-
PT(i+1)d FT(i)a*
p4, p5 p1, p2, p7
p4, p5
p3, p7
p5
p7
p5
p7
FT(i)d*
p6, p8
p6, p8
p4, p8
p8
 Compute a rule that matches the maximum number of failed test
─ The new rule r2 with accept decision can be computed as
F1  [6, 7]  F2  [7, 9]  a
32/29
Decision Fixing
 The idea of decision fixing is that for each rule ri
─ Find the passed and failed tests whose first-match rule is ri
● The set of the passed tests for ri can be computed as PT(i)-PT(i+1)
● The set of the passed tests for ri can be computed as FT(i)-FT(i+1)
─ The increased number of passed tests by change the decision of ri is
|FT(i)-FT(i+1)| - |PT(i)-PT(i+1)|
● If change ri’s decision, the passed tests in PT(i)-PT(i+1) become failed tests
and the failed tests in FT(i)-FT(i+1) become passed tests.
PT(i)-PT(i+1) FT(i)-FT(i+1)
r2: F1  [1, 5]  F2  [1, 10]  a
p1, p2
r2: F1  [1, 6]  F2  [3, 10]  a
p3
p6
r3: F1  [6,10]  F2  [1, 3]  d
p4
r4: F1  [7,10]  F2  [4, 8]  a
p8
─ Changing r4’s decision can change the failed test p8 to a passed test.
33/29
Rule Deletion
 The idea of rule deletion
─ Use all-match FDD to calculate the increased number of passed packets
F1
[1, 5]
[6, 6]
F2
[1, 2]
1,5
[7, 10]
F2
[3, 10] [1,2]
1,2,5
3,5
F2
[4,10]
[3,3]
2,3,5
2,5
[1,3]
[4,8]
3,5
4,5
[9,10]
5
 Deleting r4 changes p8 to a passed test.
─ The failed test p8 matches the first path
─ r4 and r5 have different decisions
34/29