Insecure.Org
Mastering the Nmap Scripting Engine
by Fyodor and David Fifield
http://insecure.org/presentations/BHDC10/
Black Hat Briefings Las Vegas
July 28; 4:45 PM; Augustus 5+6
Defcon 18
July 30; 5:00 PM; Track One
Insecure.Org
Outline
•
•
•
•
•
•
•
NSE Intro & Usage
Large-scale Scan #1: SMB/MSRPC
Large-scale Scan #2: Favicon
Writing NSE Scripts
Live Script Writing Demo
Nmap News
Final Notes & Q/A
Insecure.Org
Nmap Scripting Engine
# nmap -A -T4 scanme.nmap.org
Starting Nmap 5.35DC18 ( http://nmap.org )
Nmap scan report for scanme.nmap.org (64.13.134.52)
Host is up (0.0018s latency).
Not shown: 995 filtered ports
PORT
STATE SERVICE VERSION
22/tcp
open
ssh
OpenSSH 4.3 (protocol 2.0)
| ssh-hostkey: 1024
60:ac:4d:51:b1:cd:85:09:12:16:92:76:1d:5d:27:6e (DSA)
|_2048 2c:22:75:60:4b:c3:3b:18:a2:97:2c:96:7e:28:dc:dd (RSA)
53/tcp
open
domain
80/tcp
open
http
Apache httpd 2.2.3 ((CentOS))
|_html-title: Go ahead and ScanMe!
| http-methods: Potentially risky methods: TRACE
|_See http://nmap.org/nsedoc/scripts/http-methods.html
113/tcp
closed auth
31337/tcp closed Elite
OS details: Linux 2.6.13 - 2.6.31, Linux 2.6.18
Nmap done: 1 IP address (1 host up) scanned in 23.32 seconds
Insecure.Org
Pre-written Scripts and the NSEDoc Portal
http://nmap.org/nsedoc/
Insecure.Org
Script Collection Growth
140
120
100
80
60
40
20
0
2007-1
2007-2
2008-1
2008-2
2009-1
2009-2
2010-1
2010-2
Insecure.Org
Large Scale Scan #1:
SMB/MSRPC Scripts
Ron Bowes spent months researching
SMB/MSRPC protocols and wrote a suite of
13 scripts.
Informational: smb-os-discovery, smbserver-stats, smb-system-info, smb-securitymode
Detailed Enumeration: smb-enum-users,
smb-enum-domains, smb-enum-groups,
smb-enum-processes, smb-enum-sessions,
smb-enum-shares
More intrusive: smb-brute, smb-checkvulns, smb-psexec
Insecure.Org
Who to test them out on?
Insecure.Org
MS Scan Details
• Step 1: Find target IP addresses.
1,004,632 located in ARIN DB.
• Step 2: Start broad version detection scan
(nmap -T4 --top-ports 50 -sV -O --osscanlimit --osscan-guess --min-hostgroup 128
--host-timeout 10m -oA ms-vscan -iL
ms.ips.lst)
– Found 74,293 hosts up out of 1M IPs in 26 hours
• Step 3: Examine results
Insecure.Org
MS SMB Scan Results
• Vast majority of MS networks block
Windows ports such as 135 and 445 at
their gateways.
• ... but not all!
• New scan: nmap -v -O -sV -T4 --osscanguess -oA ms-smbscan --script=smbenum-domains,smb-enum-processes,smbenum-sessions,smb-enum-shares,smbenum-users,smb-os-discovery,smbsecurity-mode,smb-system-info [Target Ips]
• Results
Insecure.Org
Humble Beginnings:
The Story of http-favicon.nse
A simple idea: fingerprint web applications by
retrieving the favicon. Vlatko Kosturjak
wrote a script to do it. (http://seclists.org/nmap-dev/2008/q4/397)
However, the database was small and of
unknown quality. If only we had a tool to do
large Internet scans... and a way to write
scripts for it... (http://seclists.org/nmap-dev/2009/q3/462)
The favicon-survey.nse script (not part of the
Nmap package) downloads favicons and
stores them in the filesystem.
Insecure.Org
Results of the Favicon Survey
Scanned the external links of
•
•
•
•
•
dmoz.org: 5,042,341
en.wikipedia.org: 3,218,826
de.wikipedia.org: 832,521
fr.wikipedia.org: 652,040
es.wikipedia.org: 532,951
Omitting duplicates, around 8 million domains.
• 995,152 unique icons
• 799,924 image files
• 195,228 non-image files (HTML error pages)
“Indeed, I have been scanning ;-)”
—Brandon Enright (http:seclists.org/nmap-dev/2009/q3/487)
Insecure.Org
Favicon Visualization
A secondary survey: The Alexa top one
million sites. Pack the icons tightly, with the
size of each one proportional to its “reach.”
http://nmap.org/favicon/
Insecure.Org
Writing NSE Scripts
Insecure.Org
Introduction to Lua & Why We Chose It
• Lightweight embeddable scripting language
– Easy to learn
– Tiny to embed: “Complete distribution (source code,
manual, plus binaries for some platforms) fits
comfortably on a floppy disk”.
• Widely used, known, and debugged
– Created in Brazil in 1993, still actively developed
– Best known for its use in the game industry: World of
Warcraft, Crysis, etc.
– Security tools: Nmap, Wireshark, Snort 3.0
Insecure.Org
Why We Chose Lua (Continued)
• Extensible
– Hooked to Nmap's fast parallel networking libraries
• Safe & Secure
– No buffer overflows, format string vulns, etc.
• Portable
– Windows, Linux, Mac, *BSD, etc.
• Interpreted
Insecure.Org
Capabilities Added by Nmap
• Protocol/helper libraries
– 45, including DNS, HTTP, MSRPC, Packet,
SNMP, unpwdb, etc.
• Protocol brute forcers
• Easy SSL
• Dependencies
Insecure.Org
Script Example #1: rpcinfo
Insecure.Org
Script Example #2: smb-enum-users
Insecure.Org
Live Script Demonstration
Problem: Find my webcam on a dynamic
IP address.
The webcam uses thttpd to serve
/cam.jpg, so use a script to check those
two things.
Insecure.Org
Make it a Production Script
To turn http-brute into distribution-ready
script, I would next
• expand the portrule to match more HTTP
services,
• add script arguments to control the path
retrieved and the method used,
• add NSEDoc @usage and @output
examples, and
• let it cache credentials for other scripts to
use.
Insecure.Org
What's Coming in NSE?
• Prerules & Postrules
• Target Acquisition Scripts
• Lots more scripts! Current queue:
–
–
–
–
–
–
Vnc-info (Patrik Karlsson)
Vnc-brute (Patrik Karlsson)
Svn-brute (Patrik Karlsson)
Hostmap (Ange Gutek)
Http-xst (Eduardo Garcia Melia)
Rmi-dumpregistry (Martin Swende)
Insecure.Org
Zenmap NSE Integration
Insecure.Org
Insecure.Org
Ndiff
# ndiff facebook-072410.xml facebook-072510.xml
69.63.176.68:
PORT
STATE SERVICE VERSION
-80/tcp open http
lighttpd 1.5.0
+80/tcp open http
nginx
video-ssl-03-06-ash1.fbcdn.net (69.63.186.53):
PORT
STATE SERVICE VERSION
-443/tcp open ssl/http lighttpd 1.5.0
+443/tcp open ssl/http nginx
legacymail.thefacebook.com (66.220.144.49):
PORT
STATE SERVICE VERSION
443/tcp open ssl/http Microsoft IIS httpd 6.0
| html-title: Document Moved
-|_ Did not follow redirect to
https://mail.thefacebook.com/exchange
+|_ Did not follow redirect to
https://mail.thefacebook.com/exchange/
Insecure.Org
Simple Ndiff Cron Script
#!/bin/sh
date=`date "+%s"`
cd /hack/facebook/scripts/
nmap -T4 -F -sV -O --osscan-limit --osscan-guess
-oA facebook-${date} [netblocks] > /dev/null
ndiff facebook-old.xml facebook-${date}.xml >
facebook-diff-${date}
cp facebook-${date}.xml facebook-old.xml
printf "\n********** NDIFF RESULTS **********\n"
cat facebook-vscan-diff-${date}
printf "\n********** SCAN RESULTS **********\n"
cat facebook-vscan-${date}.nmap
Insecure.Org
Ncat
http://nmap.org/ncat/
Insecure.Org
Modern Networking Features
SSL encryption support (client or server)
Proxy (act as proxy server, or client
chaining through multiple proxies )
Portability
TCP/UDP port redirection
IPv6
Fine-grained access control
Connection brokering
Missing feature
Insecure.Org
Ncat Chat
A slight hack to broker mode enables a
very rudimentary chat server.
Official chat server for this presentation:
ncat --ssl -v chat.nmap.org
Server was started with command:
ncat -l --ssl --chat chat.nmap.org
Insecure.Org
Insecure.Org
Insecure.Org
Rainmap: An Online Scanning Service
Insecure.Org
Nmap Script Authors
Aaron Leininger
Andrew Orr
Ange Gutek
Arturo Busleiman
Bernd Stroessenreuther
Brandon Enright
David Fifield
Diman Todorov
Djalal Harouni
Doug Hoyte
Duarte Silva
Eddie Bell
Eugene V. Alexeev
Felix Groebert
Ferdy Riphagen
Jah
Jason DePriest
Joao Correa
Kris Katterjohn
Mak Kolybabi
Marek Majkowski
Martin Swende
Matthew Boyle
Michael Pattrick
Michael Schierl
Patrik Karlsson
Philip Pickering
Richard Sammet
Rob Nicholls
Ron Bowes
Sven Klemm
Thomas Buchanan
Tom Sellers
Vladz
Vlatko Kosturjak
Insecure.Org
Final Notes
• Ron Bowes at Black Hat Arsenal
– Thursday, Station 5, 8:00 AM – 12:30 PM
• Slides to be posted Friday (and video as
soon as we get it) to:
http://insecure.org/presentations/
• Download Nmap from: http://nmap.org
• NSEDoc portal: http://nmap.org/nsedoc/
• NSE system docs:
http://nmap.org/book/nse.html
Insecure.Org
Questions?