NetWitness Investigator
Freeware
Network Intelligence, Threat Indicators
and Session Exploitation
Brian Girardi
Director, Product Management
NetWitness Corporation
brian@netwitness.com
1
| Copyright 2010 © All rights reserved. NetWitness Corporation
Agenda
» Investigator Freeware Introduction/Review
» Advanced Features
»
2
‣ Integration via “custom actions”
‣ Intelligence via “feeds”
‣ Indicators via “rules”
‣ Protocol/Session exploitation via “parsers”
Implementation Scenarios
| Copyright 2010 © All rights reserved. NetWitness Corporation
Investigator Freeware Core Concepts
» Its free! – requires annual registration
» What makes Investigator different?
‣ Designed from an analysts perspective to answer complex
questions from large amounts of raw network data
‣ Designed to analyze advanced threats, applications, content,
incident response, <insert problem here>
‣ Empowers novice analysts AND accelerates experts
‣ Models network traffic, and exposes syntax to expand the
model
‣ Session-based NOT packet-based
3
| Copyright 2010 © All rights reserved. NetWitness Corporation
Session Processing Step 1
Packet Collection & Reassembly before anything else
•
data
putting the pieces back together
packetized
Retransmitted
out of order
fragmented
Mixed with other traffic
x
Session
4
| Copyright 2010 © All rights reserved. NetWitness Corporation
Session Processing steps 2 & 3
»
Application Identification, Meta Extraction, and Modeling
•
•
•
Don’t rely on port for true service type
Extract pertinent network and application data
Model and organize data for human consumption
HTTP != port 80
5
| Copyright 2010 © All rights reserved. NetWitness Corporation
Standard Features
» Real-time, patented layer 7 analytics
»
»
»
»
– Effectively analyze data starting from
application layer entities like
users, email, address, files ,
and actions.
– Infinite, free-form analysis paths
– Content starting points
Captures raw packets live from wired or
802.11 wireless networks
Imports packets from any open-source,
home-grown and commercial packet capture
system (e.g. .pcap file import)
Extensive network and application layer
filtering (e.g. MAC, IP, User, Keywords,
Etc.)
IPv6 support
» Full content search, with Regex support
» Bookmarking & history tracking
» Integrated GeoIP for resolving IP addresses
»
»
»
»
»
»
»
»
to city/county, supporting Google® Earth
visualization
SSL Decryption (with server certificate)
Interactive time charts, and summary view
Interactive packet view and decode
Hash data on capture and export
Integrated Org, Domain, and ISP
databases
Supports VLAN meta tagging
Supports IP Tunnel (i.e. GRE) meta tagging
And More….
Now lets discuss advanced features…
6
| Copyright 2010 © All rights reserved. NetWitness Corporation
Apply Your Own Intelligence & Needs
»
»
»
Custom Actions
‣ “Right-click” query actions for context
Feeds
‣ Means for creating meta data based on a list of values
‣ Ex. IP Reputation Feed
Rules
‣ Evaluation of meta elements to alert, filter, stop/change processing or create
more metadata
‣ Ex. If ip.dst=1.2.3.4 AND user=‘bob’ then alert
»
Parsers (aka FlexParse™)
‣ Exploitation of sessions and full payload to create metadata
‣ Ex. Identify packed executables/malware, interpret identify and profile
protocols.. Etc.
7
| Copyright 2010 © All rights reserved. NetWitness Corporation
Aggregating Indicators
Aggregation of these
methods help profile
actual threatening activity
• Advanced Threat
• Insider Threat
• Policy/Compliance
• Etc.
Feeds
Parsing
8
| Copyright 2010 © All rights reserved. NetWitness Corporation
Rules
Custom Actions
9
| Copyright 2010 © All rights reserved. NetWitness Corporation
Custom Actions
» Configurable “right-click” actions out of Investigator to external tools
»
10
‣ URL-based
‣ Local Scripts
Examples
| Copyright 2010 © All rights reserved. NetWitness Corporation
Example: right-click hostname into Google
Other options…
11
| Copyright 2010 © All rights reserved. NetWitness Corporation
Feeds
12
| Copyright 2010 © All rights reserved. NetWitness Corporation
Feeds
» Means for creating meta data based on external lists
»
‣ IP Address
‣ Hostnames
‣ Any metadata element
Typical Uses
‣ Intelligence Feeds ( Internet Storm Center/Dshield Top 10000 for
example)
‣ Define Physical or Logical mappings for metadata
•
•
•
•
•
13
Campus, Department
User Identity via Active Directory
Network-specific maps
DHCP mappings
Etc…
| Copyright 2010 © All rights reserved. NetWitness Corporation
Real-world feed uses
» Large Bank
•
17,000 known Home User IPs cross-referenced with botnet membership list
•
4000+ subnets, largely model after base locations
» DOD
» Financial Services Firm
•
•
•
14
Buildings
Functional Area ie: Network Infrastructure
System Area ie: Firewall, VPN, Critical Servers
| Copyright 2010 © All rights reserved. NetWitness Corporation
Department & Location Feed
» Enterprise-specific context
‣ IP Ranges that correlate to
•
•
•
Company Department
Physical Location
Lat/Long Override
» Feed File Example
#networks#
172.16.60.1,172.16.60.254,NW-Wireless
172.16.70.1,172.16.70.254,NW-GuestNet
10.21.1.1,10.21.1.255,NW Infrastructure,38.967490,-77.379533
10.21.2.30,10.21.2.111,NW Users Net,38.967490,-77.379533
10.21.3.30,10.21.3.111,NW Dev Workstations,38.967490,-77.379533
10.21.4.1,10.21.4.255,NW Dev Servers,38.967490,-77.379533
10.21.5.1,10.21.5.111,NW VPN Users,38.967490,-77.379533
10.21.6.30,10.21.6.111,NW Wireless,38.967490,-77.379533
67.10.149.25,67.10.149.25,Nw TXGW,29.7296,-98.1001
172.16.55.0,172.16.55.255,NW-TX,29.7296,-98.1001
172.16.55.0,172.16.55.255,NW-TX,29.7296,-98.1001
192.168.1.1,192.168.1.255,NW Lab,38.742641,-77.199997
15
| Copyright 2010 © All rights reserved. NetWitness Corporation
Feed Definition File
<FlatFileFeed name="NetName" path="networks.txt" separator="," comment="#">
<LanguageKeys>
<LanguageKey name="netname" valuetype="Text"
srcname="netname.src" destname="netname.dst"/>
</LanguageKeys>
<Fields>
<Field index="1" type="index" range="low"/>
<Field index="2" type="index" range="high"/>
<Field index="3" type="value" key="netname"/>
</Fields>
</FlatFileFeed>
16
| Copyright 2010 © All rights reserved. NetWitness Corporation
Netname Feed Classification
17
| Copyright 2010 © All rights reserved. NetWitness Corporation
Analysis with Threat Feeds
18
| Copyright 2010 © All rights reserved. NetWitness Corporation
Loading Internet Storm Center Feed
Load feeds
19
| Copyright 2010 © All rights reserved. NetWitness Corporation
Feed Category Hits
Found hits on SANS feed
20
| Copyright 2010 © All rights reserved. NetWitness Corporation
Session Details Review
HTTP put
Likely C&C querystring
IP Found in SANS feed
Encoded/Encrypted payload
21
| Copyright 2010 © All rights reserved. NetWitness Corporation
Rules
22
| Copyright 2010 © All rights reserved. NetWitness Corporation
Rules
» Rules can be used to
»
»
23
‣ filter in/out data
‣ truncate packets
‣ alert/flag
Rules span
‣ network elements
‣ application layer elements
Control depth of processing
| Copyright 2010 © All rights reserved. NetWitness Corporation
Network Layer Rules
24
| Copyright 2010 © All rights reserved. NetWitness Corporation
Application Layer Rules
25
| Copyright 2010 © All rights reserved. NetWitness Corporation
Rule Examples
» Filter
»
»
26
‣ Advertisements (ends in “doubleclick.net”)
‣ Software Updates (ends in “liveupdate.symantec.com”)
‣ Media (ends in “player.xmradio.com”)
‣ Backup servers (192.168.1.54…etc)
‣ Filter *(All), Keep email = “scott4323@hotmail.com”
Truncate
‣ Drop packet payload for port SSH and SSL
Alert
‣ Non-standard port activity (non-HTTP over port 80)
‣ DynDNS Domains
‣ BOT Profiles
‣ Clear text passwords
‣ Tunneling services ( gotomypc, anonymizers, etc. )
‣ Specific threat profiles
‣ Etc…etc…etc…
| Copyright 2010 © All rights reserved. NetWitness Corporation
Rule Example
Tip: faster to check
range than !=
27
| Copyright 2010 © All rights reserved. NetWitness Corporation
Non-standard HTTP
28
| Copyright 2010 © All rights reserved. NetWitness Corporation
Nonstandard HTTP Details
29
| Copyright 2010 © All rights reserved. NetWitness Corporation
Facebook Koobface Malware Example
» Basic Rule:
»
‣ Service = HTTP(80) && alias.host = ‘locator.getconnected.be’
Better Rule:
‣ Service = HTTP(80) && alias.host exists && (query contains 'action='
&& query contains 'c_fb=' && query contains 'c_ms=' && query contains
'c_hi=' && query contains 'c_tw=' && query contains 'c_be=' && query
contains 'c_tg=' && query contains 'c_nl=’)
» Based on the url parameters koobface passes when it checks in
Ref: http://us.trendmicro.com/imperia/md/content/us/trendwatch/researchandanalysis/the_20heart_20of_20koobface_final_1_.pdf
30
| Copyright 2010 © All rights reserved. NetWitness Corporation
Parsers
31
| Copyright 2010 © All rights reserved. NetWitness Corporation
FlexParse™
»
FlexParse exposes the network session parsing and metadata model
‣ Configure how to identify applications and extract data
•
•
•
•
»
Register search tokens
Perform logic operations
Register metadata for the NetWitness system
Why?
‣ Instantly customize and expand processing and modeling behavior
‣ Processing flexibility for networks with:
•
•
•
»
XML parser definitions
heavy application profiles
proprietary protocols
and threats that don’t fall into common intrusion detection methods
What's possible…
‣ Expand baseline parsers, fast flux identification, social networking
profiling, mainframe exploitation, SCADA, file object identification,
complex threat identification, …Etc.
32
| Copyright 2010 © All rights reserved. NetWitness Corporation
Copyright 2007 NetWitness Corporation
SCADA MODBUS Parser
33
| Copyright 2010 © All rights reserved. NetWitness Corporation
Simple MODBUS Parser
»
Why?
‣ Need insight into SCADA over IP to correlate with other network activity –
critical infrastructure monitoring
»
Demonstrate
‣ Create new Service type for MODBUS
‣ Simple text based protocol has numeric tokens that map to actions:
•
•
•
•
•
•
•
34
“Read Coil Status”
“Read Input Status”
“Read Hold Registers”
“Read Input Registers”
“Force Single Coil”
“Force Multiple Coils”
Etc……
| Copyright 2010 © All rights reserved. NetWitness Corporation
MODBUS Protocol
» If port 502 AND tokens exist then classify and extract actions --» Request
MODBUS
ACTION
PROTOCOL
35
| Copyright 2010 © All rights reserved. NetWitness Corporation
Simple MODBUS protocol FlexParser Syntax
<?xml version="1.0" encoding="utf-8"?>
<parsers xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="parsers.xsd”>
<parser name="MODBUS" desc="MODBUS SCADA Protocol" service="502">
<declaration>
<number name="vTemp" />
<number name="vState" />
<number name="vID"/>
<port name="server-port" value="502" />
<meta name="action" format="Text" key="action"/>
</declaration>
<match name="server-port”>
<assign name="vTemp" value="1" />
<while name="vTemp" equal="1”>
<assign name="vTemp" value="0" />
<move value="2”>
<read name="vState" length="2”>
<if name="vState" equal="0”>
<assign name="vID" value="1" />
<assign name="vTemp" value="1" />
<move value="3”>
<read name="vState" length="1”>
<if name="vState" equal="1”>
<register name="action" value="Read Coil Status"/>
</if>
<if name="vState" equal="2”>
<register name="action" value="Read Input Status"/>
</if>
<if name="vState" equal="3”>
<register name="action" value="Read Hold Registers"/>
</if>
<if name="vState" equal="4”>
<register name="action" value="Read Input Register"/>
</if>
…………….
36
| Copyright 2010 © All rights reserved. NetWitness Corporation
Detecting Malicious PDF Parser
37
| Copyright 2010 © All rights reserved. NetWitness Corporation
Detecting Malicious PDFs
» Why?
‣ One of the most pervasive exploitation techniques used currently
‣ Very effective exploitation technique that can be difficult to detect
» Demonstrate
‣ Combined existence of PDF tokens, including javascript that classifies
potentially malicious objects
‣ Use “flags” to keep “state” between several different <match>
statements
38
| Copyright 2010 © All rights reserved. NetWitness Corporation
Parser Logic
» Find the following token:
»
»
»
39
‣ HTTP/1.1 200 OK
If above is found, then find token:
‣ Content-Type: application/pdf
If above is found, then find token:
‣ %PDF-1.
If above is found, then alert if the following is found:
‣ /S/JavaScript
| Copyright 2010 © All rights reserved. NetWitness Corporation
Parser Syntax
<declaration>
<token name="token_http_header" value="HTTP/1.1 200 OK"
options="linestart"/>
<token name="token_content_type" value="Content-Type: application/pdf"
options="linestart"/>
<token name="token_pdf_header" value="%PDF-1."/>
<token name="token_open_brackets" value="&#x3C;&#x3C;"/>
<number name="flag_state_traker" scope="session"/>
<string name="str_holding"/>
<number name="num_offset"/>
<meta name="event" key="alert" format="Text"/>
</declaration>
40
| Copyright 2010 © All rights reserved. NetWitness Corporation
Declare tokens
Parser Syntax
<match name="token_http_header">
<assign name="flag_state_traker" value="1"/>
</match>
Maintain state
of token identification
<match name="token_content_type">
<if name="flag_state_traker" equal="1">
<assign name="flag_state_traker" value="2"/>
</if>
</match>
<match name="token_pdf_header">
<if name="flag_state_traker" equal="2">
<assign name="flag_state_traker" value="3"/>
</if>
</match>
41
| Copyright 2010 © All rights reserved. NetWitness Corporation
Parser Syntax
<match name="token_open_brackets">
<if name="flag_state_traker" equal="3">
<find value="&#x3E;&#x3E;" length="50" name="num_offset">
<read length="$num_offset" name="str_holding">
<find in="$str_holding" name="num_offset" value="S/JavaScript">
<register name="event"
value="lab_advanced_pdf_with_javascript"/>
</find>
</read>
</find>
</if>
</match>
Find javascript in PDF
42
| Copyright 2010 © All rights reserved. NetWitness Corporation
Suspicious Trigger
Parser
alert
43
| Copyright 2010 © All rights reserved. NetWitness Corporation
PDF with Javascript
Matched
tokens
44
| Copyright 2010 © All rights reserved. NetWitness Corporation
JRE 0day Analysis … the short version
Using Feeds, Rules & Parsers to Investigate & Profile
45
| Copyright 2010 © All rights reserved. NetWitness Corporation
Background
»
April 9th 2010 – Tavis Ormandy of Google
Security identifies Java Deployment Toolkit
flaw
‣ Affects all versions of Java
»
April 11th Active exploitation via Rogue
Advertisements on nytimes.com,
foxnews.com, oprah.com, ufc.com and
others
‣ Malicious .jar file
‣ Referrers contains
‘nytimes.com’,’foxnews.com’,
’oprah.com’,ufc.com’
»
How do we leverage feeds, rules and
parsers to profile? Do I have a problem?
‣ 0day, feeds may not provide intelligence
46
| Copyright 2010 © All rights reserved. NetWitness Corporation
Hunting for Anomalous Traffic
» Profile HTTP for java-archives (potential deployment toolkit)
» Rule: service = HTTP(80) && content = ‘application/java-archive’
Dig more on this…
47
| Copyright 2010 © All rights reserved. NetWitness Corporation
Internal host being referred to what?
»
Use IP from anomalous traffic analysis
‣ Rule: ip.src = 156.145.x.x && referrer contains
‘nytimes.com’,’foxnews.com’,etc..’
»
Redirection to 95.211.14.21
‣ Netherlands Hosting Provider
‣ 95.211.14.21/measure/ad.php
‣ Inspect php
»
Rule to profile & find ad.php querystring:
‣ service = HTTP(80) && (query contains 'pl=' &&
query contains 'ce=' && query contains 'hb=' &&
query contains 'av=' && query contains 'jv=’)
48
| Copyright 2010 © All rights reserved. NetWitness Corporation
Ad.php behavior
» Downloads “p.gif” from referred location
Really?
.gif?
» How many times have I seen this “.gif”?
49
| Copyright 2010 © All rights reserved. NetWitness Corporation
Compromised Hosts
» Rule: service = HTTP(80) && filename=‘p.gif’ && content =
‘application/octet-stream’
» 3 Sessions
» 3 Unique hosts
50
| Copyright 2010 © All rights reserved. NetWitness Corporation
Deeper Analysis…
Huh?
»
»
p.gif (exe) appears corrupt
‣ Does that mean no one was infected?
Let’s have a look at the .jar
MZ
»
»
51
.jar modifies the first two bytes of the binary to subvert “MZ” token
signatures
FlexParse profile the malware…
| Copyright 2010 © All rights reserved. NetWitness Corporation
Flex Parser for Obfuscated Exe in Image
<parser name="non_matching_app_content_type" desc="non_matching_app_content_type">
<declaration>
<meta name="alert" key="alert" format="Text"/>
<token name="get" value="GET " options="linestart"/>
<token name="content" value="This program cannot be run in DOS mode"/>
<token name="content" value="This program must be run under Win32"/>
<token name="named_types" value=".jpg HTTP/1.1" options="linestop"/>
<token name="named_types" value=".gif HTTP/1.1" options="linestop"/>
<token name="named_types" value=".png .....<snip>
<number name="session_flag" scope="session"/>
</declaration>
<match name="get">
<assign name="session_flag" value="0"/>
If GET image & content contains
</match>
“… run in DOS mode…”
<match name="named_types">
<if name="session_flag" equal="0">
“… under Win32…”
<assign name="session_flag" value="2"/>
</if>
</match>
<match name="content">
<if name="session_flag" equal="2">
<register name="alert" value="non_matching_app_content_type"/>
<assign name="session_flag" value="0"/>
</if>
</match>
52
| Copyright 2010 © All rights reserved. NetWitness Corporation
Summary
»
»
»
»
Investigator – Free!
Custom actions, Feeds, Rules and Parsers expand to expand analytical
capabilities
Aggregating advanced indicators and profiling techniques really help
Resources
‣ Community (http://community.netwitness.com)
•
•
•
•
Rule examples
FlexParser examples
Tips/Tricks
Discussion
‣ YouTube (http://www.youtube.com/netwitness)
‣ Training Webcasts ( www.netwitness.com )
‣ Brian Girardi, brian@netwitness.com
53
| Copyright 2010 © All rights reserved. NetWitness Corporation
Q&A
54
| Copyright 2010 © All rights reserved. NetWitness Corporation