The Bro Network Security Monitor
Overview and Recent Developments
Outline
• Philosophy and Architecture
- A framework for network traffic analysis
• History
- From research to operations
• Architecture
- Components, logs, scripts, cluster
2
What is Bro?
Packet Capture
Traffic Inspection
Attack Detection
“Domain-specific Python”
NetFlow
syslog
Log Recording
Flexibility
Flexibility
Abstraction
Abstraction
Data Structures
Data Structures
3
Philosophy
•Fundamentally different from other IDS
– Reset your idea of an IDS before starting to use Bro
•Real-time network analysis framework
– Primarily an IDS, but many use it for general traffic
analysis
•Policy-neutral at the core
– Can accommodate a range of detection approaches
4
Philosophy (cont’d.)
•Highly stateful
– Tracks extensive application-layer network state
•Supports forensics
– Extensively logs what it sees
5
Target Audience
•Large-scale environments
– Effective also with liberal security policies
•Network-savvy users
– Requires understanding of your network
•Unixy mindset
– Command-line based, fully customizable
6
Research Heritage
•Much of Bro is coming out of research projects
– Bridging gap between academia and operations
•However, that meant limited engineering
resources
– We were lacking resources for development,
documentation, polishing
Office of Cyberinfrastructure
7
Research Heritage (cont’d.)
•NSF now funding Bro development at ICSI and
NCSA
– Full-time engineers working 3 years on capabilities
& user experience
•Objective is a sustainable development model
– Aiming to create a larger user and development
community
Office of Cyberinfrastructure
8
Bro History
Host Context
Time Machine
Enterprise Traffic
TRW
State Mgmt.
Independ. State
Stepping
Stone
Detector
USENIX
Paper
199
5
199
6
Vern writes
1st line of
code
199
7
v0.2
1st
CHANGES
entry
LBNL starts
using Bro
199
8
199
9
200
0
Anonymizer
Active Mapping
Context Signat.
200
1
200
2
200
3
200
4
2nd Path
BinPAC
DPD
200
5
200
6
Parallel
Prototype
Autoconf
200
7
200
8
200
9
201
0
201
1
Bro
v1.1/v1.2
v0.8aX/0.9aX
v1.5
2.0
when
SSL/SMB
BroControl
Resource
STABLE releases
tuning
BroLite
Bro
Broccoli
Waters
v1.0 DPD
v1.4
v0.7a175/0.8aX
BinPAC
DHCP/BitTorrent
Signatures
IRC/RPC
HTTP entities
SMTP
analyzers
NetFlow
IPv6 support
64-bit support
Bro Lite
User manual
Sane version
Deprecated
numbers
0.8a37
v1.3
v0.7a48
Communication
Ctor expressions
Consistent
Persistence
GeoIP
CHANGES
Namespaces
Conn Compressor
Log Rotation
v0.7a90
v0.6
Profiling
RegExps
State Mgmt
Login analysis
v0.4
HTTP
analysis
Scan
detector
IP fragments
Linux
support
Bro Cluster
Shunt
9
Deployment
Tap
Internet
Bro
Runs on commodity platforms
Standard PCs & NICs
Supports FreeBSD/Linux/OS X.
10
Internal
Network
Architecture
Logs
Analysis Logic
Notification
Policy Script Interpreter
Events
Protocol Decoding
Event Engine
Packets
Network
11
Event Model
Request for /index.html
Web Client
1.2.3.4/4321
.
.
.
Status OK plus data
1.2.3.4/4321
SYN SYN
Event
ACK
Stream of TCP
packets
ACK
.
.
.
Web
Server
ACK
ACK
5.6.7.8/80
FIN
FIN
connection_established(1.2.3.4/4321⇒5.6.7.8/80)
Event
TCP stream reassembly for
originator
http_request(1.2.3.4/4321⇒5.6.7.8/80, “GET”, “/index.html”)
TCP stream reassembly for
responder
Event
http_reply(1.2.3.4/4321⇒5.6.7.8/80, 200, “OK”, data)
connection_finished(1.2.3.4/4321, 5.6.7.8/80)
Event
12
Script Example: Matching URLs
Task: Report all Web requests for files called “passwd”.
event http_request(c: connection,
method: string,
original_URI: string,
unescaped_URI: string,
version: string)
{
if ( method == "GET" && unescaped_URI
NOTICE(...); # Alarm.
}
13
#
#
#
#
#
Connection.
HTTP method.
Requested URL.
Decoded URL.
HTTP version.
== /.*passwd/ )
Script Example: Scan Detector
Task: Count failed connection attempts per source address.
global attempts: table[addr] of count &default=0;
event connection_rejected(c: connection)
{
local source = c$id$orig_h;
#
local n = ++attempts[source];
#
if ( n == SOME_THRESHOLD )
#
NOTICE(...);
#
}
14
Get source address.
Increase counter.
Check for threshold.
Alarm.
Distributed Scripts
•Bro comes with >10,000 lines of script code
– Prewritten functionality that’s just loaded
•Scripts generate alarms and logs
– Amendable to extensive customization and
extension
15
Example Logs
> bro -i en0
[ ... wait ...]
> cat conn.log
#fields ts
1144876741.1198
1144876612.6063
1144876596.5597
1144876606.7789
1144876741.4693
1144876745.6102
1144876605.6847
id.orig_h
192.150.186.169
192.150.186.169
192.150.186.169
192.150.186.169
192.150.186.169
192.150.186.169
192.150.186.169
id.orig_p
53115
53090
53051
53082
53116
53117
53075
id.resp_h
82.94.237.218
198.189.255.82
193.203.227.129
198.189.255.73
82.94.237.218
66.102.7.99
207.151.118.143
id.resp_p
80
80
80
80
80
80
80
proto
tcp
tcp
tcp
tcp
tcp
tcp
tcp
service
http
http
http
http
http
http
http
duration
16.14929
4.437460
0.372440
0.597711
16.02667
1.004346
0.029663
obytes
435
8661
461
337
3027
422
347
rbytes [...]
66363
63663
753
5146
11761
1637
1011
> cat http.log
#fields ts
1144876741.6335
1144876742.1687
1144876741.2838
1144876742.3337
1144876742.3337
1144876742.3337
1144876742.3337
1144876742.3338
1144876745.6144
id.orig_h
192.150.186.169
192.150.186.169
192.150.186.169
192.150.186.169
192.150.186.169
192.150.186.169
192.150.186.169
192.150.186.169
192.150.186.169
id.orig_p [...] host
53116
docs.python.org
53116
docs.python.org
53115
docs.python.org
53116
docs.python.org
53116
docs.python.org
53116
docs.python.org
53116
docs.python.org
53116
docs.python.org
53117
www.google.com
16
uri
/lib/lib.css
/icons/previous.png
/lib/lib.html
/icons/up.png
/icons/next.png
/icons/contents.png
/icons/modules.png
/icons/index.png
/
status_code
200
304
200
304
304
304
304
304
200
user_agent [...]
Mozilla/5.0
Mozilla/5.0
Mozilla/5.0
Mozilla/5.0
Mozilla/5.0
Mozilla/5.0
Mozilla/5.0
Mozilla/5.0
Mozilla/5.0
Bro Ecosystem
Time Machine
Bro Distribution
Tap
Tap
Internal
Network
Internet
bro-2.0.tar.gz
Contributed
Scripts
Functionality
Bro
Control
Events
State
Output
Events
bro-aux
BinPAC
capstats
Bro Client Communication Library
BTest
tracesummary
BroControl
Broccoli Python
Broccoli
Broccoli Ruby
User Interface
(Broccoli Perl)
http:://www.bro-ids.org/download
git://git.bro-ids.org
17
Bro Cluster Ecosystem
Time Machine
Tap
Tap
Internal
Network
Internet
LoadBalancer
Packets
Contributed
Scripts
Functionality
Bro
“Frontend”
Bro
Bro
Bro
Events
State
“Workers”
Control
bro-aux
BTest
BinPAC
tracesummary
Output
Events
capstats
“Manager”
Bro Client Communication Library
BroControl
Broccoli Python
Broccoli
Broccoli Ruby
User Interface
(Broccoli Perl)
18
Agenda
19
“The Bro Team”
20