The Evolution of Network Configuration:
A Tale of Two Campuses
Hyojoon Kim†, Theophilus Benson‡
Aditya Akella‡, Nick Feamster†
†Georgia Tech
‡University of Wisconsin, Madison
1
What is Network Configuration?
• Collection of
configuration files
• Express network policy
• Determines the overall
network behavior
2
The Network State Changes
• Topology change
• Policy change
Configuration change
Growth of firewalls in Georgia Tech
How does network configuration
change over time?
3
Configuration Changes
of line
WhatGeorgia
are Tech
causingNumber
the changes?
Network Devices
Routers
Firewalls
Switches
changes
326,458
539,171
353,420
Where are the changes happening?
Total
1,219,049
Line changes in the past 5 years
Is there a noticeable pattern?
4
Our Contribution
• Examine change patterns over time
• Look at many different types of devices
• Provide better understanding
– Help develop better configuration tools
• e.g., Change recommendations, feedbacks
– Reduce misconfigurations
5
Our Data
• Configuration data from two campus
networks
– 5 years of accumulated configuration files
• Tools
– CVS
– RANCID (Really Awesome New Cisco confIg Differ)
6
Collecting Configuration Files
…
Pull configuration
Remote login
(telnet, ssh)
RANCID
CVS commit
CVS Server
7
Revision Control on Configuration Files
• When is the change?
• What changed?
• Regenerate each
revision
...
RCS Format
1.51
log
@Fri Feb 5 15:04:28 EST 2010
@
text
@a141 1
port-object range bootps bootpc
a160 4
object-group service 12-123-12-13-any-udp udp
port-object range bootps bootpc
object-group service 12-123-12-14-any-udp udp
port-object range bootps bootpc
d173 16
a188 9
object-group service 13-14-15-16-any-udp udp
port-object range bootps bootpc
object-group service 14-15-16-17-any-udp udp
...
8
Our Approach
Take latest
snapshot
Compare revisions
Data
(RCS)
Revisions
Sort revisions
by time
Group
simultaneous
changes
Snapshot
Analysis
Change
Analysis
Longitudinal
Analysis
Correlation
Analysis
9
Classifying Configuration lines
logging buffered 1024000
enable secret [deleted]
username [deleted]
aaa new-model
…
Interface Port-channel1
description WiSM-A virtual channel
switchport trunk encapsulation dot1q
switchport trunk allowed vlan 316,805,807-809,816,1296,1312
switchport mode trunk
…
router ospf xxxx
router-id x.x.x.x
…
ip access-list extended access-vty-in
permit tcp x.x.0.0 0.0.255.255 any range 22 telnet log-input
permit tcp x.x.0.0 0.0.255.255 any range 22 telnet log-input
…
Management
Layer 1
Layer 2
VLAN
Layer 3
ACL
Security
Control Filter
QoS
10
Overview of Results
• Routers are multi-functional
– Univ. of Wisc: Layer 3 changes are 30% of total changes
– Georgia Tech: Layer 3 changes are 5% of the total changes
• Firewall changes are concentrated on ACL
– Around 87% of the total changes
– Steep increase in the access control list lines
• Switches are about providing connectivity
– Port-centric changes
11
Change Analysis on Routers
78%
Static
ARP
Number of line changes in all routers over 5 years - GT
12
Change Analysis on Firewalls
87%
Access
Control
Number of changes in all Georgia Tech firewalls over 5 years
13
Longitudinal Analysis on Firewalls
Change in number of Lines in
all Georgia Tech firewalls
Change in number of firewalls
in Georgia Tech
14
Change Analysis on Switches
snmp trap
Number of line changes in all switches in Univ. of Wisconsin
15
Correlation Analysis on Switches
Univ. of Wisconsin Switches
Correlated changes
ACL, L1
L1, VLAN
L1, L2, MGT
MGT, L1
VLAN, MGT
%
24%
11%
11%
10%
9%
16
Conclusion
• Study on how network configuration
changes over time
• Reveal interesting characteristics about
network changes
– Magnitude and frequency of changes
– Causes of changes
17
Conclusion
• Provide better understanding
• Improve current methods of configuring and
managing network devices
– Change recommendations
– Reduce misconfigurations
– More automation
Questions?
joonk@gatech.edu
18
Georgia Tech Network
Routers
Firewalls
Switches
Total
16
365
716
1,097
19