GIMS Tutorial - GEC 12
Packet Capture for GENI
Charles Thomas
& Dongchan Kim
University of Wisconsin - Madison
cthomas@wisc.edu
dkim24@wisc.edu
GIMS Tutorial
• Follow along (cut and paste commands):
http://gims.wail.wisc.edu/docs/Tutorial.html
• Start your VirtualBox Tutorial image.
Tutorial Outline
•
•
•
•
•
Introduction to GIMS.
Setting up for using GIMS.
Creating a capture filter.
Creating an RSpec.
Creating a slice and starting the experiment
Tutorial Outline (Cont.)
•
•
•
•
•
•
Tour of the monitoring GUI tools.
Pause/Restart traffic capture.
Stop and teardown the experiment.
Post-experiment data analysis.
System capabilities.
Future directions / Q&A
What is GIMS?
• GENI Instrumentation and Measurement Systems
• Collaboration between University of Wisconsin
(Barford), Colgate University (Sommers) and
Boston University (Crovella)
• High-speed packet capture system integrated with
ProtoGENI.
• Infrastructure could be modified to support other
types of instrumentation.
GIMS Architecture
• ProtoGENI Scripts
– Allow us to control the capture system.
• GIMS Backend
– Coordinates communication.
• MySQL Database
• Capture Device
– Daemon & hardware
• Web-based GUIs
– Device and configuration control, real-time
monitoring, experiment results.
GIMS Database
• MySQL
• Stores information about:
– Experiments
– Capture Devices
– Experiment/Device pairings
– Device Configurations
– Experiment Results
– Device Statistics
‘capd’, the GIMS Capture
Daemon
•
•
•
•
•
•
•
XML/RPC control structures
Supports a wide variety of hardware.
libpcap support
Flow aggregation based on IPFIX
CryptoPan anonymization
Sampling (every N, randomized)
Remote storage (SSH, Amazon S3)
ProtoGENI Scripts
Current Functionality
• CreateSliver
– Create new experiment
– Load config into device
– Start capture
• StopSliver
– Pause capture
• StartSliver
– Restart capture after pause
• DeleteSlice
– Shutdown capture activities and cleanup.
Installing the ProtoGENI Scripts
M2crypto
> sudo apt-get install python-m2crypto
Test Scripts
> wget http://www.emulab.net/downloads/protogeni-tests.tar.gz
Test Scripts
>mkdir test_scripts;
>tar xzf protogeni-tests.tar.gz –C test_scripts;
Emulab Account
Click ‘Request Account’ button on http://www.schooner.wail.wisc.edu
Emulab Account
Choose the corresponding option
Emulab Account
Fill out the form
Emulab Account
Click the ‘Submit’ button when done
SSL Certificate
Click ‘Login’ button
SSL Certificate
Login with your account
SSL Certificate
Click ‘Generate SSL Cert’ in the left column
SSL Certificate
Type the PassPhrase for your certificate and Emulab Password
SSL Certificate
Click ‘Download’ to save your SSL Certificate
SSL Certificate
Save the SSL Certificate as ‘$HOME/.ssl/encrypted.pem’
SSL Certificate
Create a ‘$HOME/.ssl/password’ file containing the PassPhrase for your
certificate
SSH Key
> cd $HOME/.ssl; ssh-keygen –f protogeni-key
SSH Key
Click ‘Edit SSH Keys’ on the Emulab login page
SSH Key
Add the public key and Emulab password at the bottom of the web page
SSH Key
The keys added to your account
Creating a Capture Filter
http://gims.wail.wisc.edu/cgi-bin/GIMSControl.cgi
Setup Capture Filter
•
•
•
•
•
•
Config Name: GEC12_<name>
Source IP address: 10.1.1.3
Destination IP address: 10.1.1.2
Source port number: 48060
Destination port number: 5001
Protocol: UDP
Setup Capture Filter (Cont.)
•
•
•
•
•
•
•
•
SSH User: gimsusr
SSH Host: ops.schooner.wail.wisc.edu
SSH Port: 22
SSH Path: /proj/GEC12/data
SSH Private Key: foo
Rollover Interval: 1 minute
Sampling: everyN
Param: 5
Creating an Rspec For Your
Experiment
cd ~geni/Tutorials/GIMS/
cp GimsMSNProduction.rspec GEC12.rspec
vi GEC12.rspec
Save as “GEC12.rspec”
When you’re done editing, simply type “ZZ”
Creating a Slice and Starting the
Experiment
cd ~geni/Tutorials/GIMS
ls -l
./registerslice.py -n <slicename>
./createsliver.py -m
https://gims.wail.wisc.edu/protogeni/xmlrpc/cm -d -n
<slicename> GEC12.rspec
GIMS Monitoring Tools
•gimsd - Device monitor
•LogViewer.cgi
•Results.cgi
‘gimsd’ - The GIMS Hardware
Monitoring Daemon
•Sits in a loop looking for running experiments.
•If it finds any, queries the capture device via
SNMP to get information on the device status
(Only queries each device once.), storing results
in DB.
•Calls ‘GetExperimentStats’ for each running
experiment and stores the results in the
database.
•Runs every 15 seconds (configurable).
Pause/Restart Capture
./sliveraction.py -m
https://gims.wail.wisc.edu/protogeni/xmlrpc/cm -d -n
<slicename> stop
./sliveraction.py -m
https://gims.wail.wisc.edu/protogeni/xmlrpc/cm -d -n
<slicename> start
Stopping and Tearing Down the
Experiment
./deleteslice.py -m
https://gims.wail.wisc.edu/protogeni/xmlrpc/cm -d -n
<slicename>
Post-Experiment Analysis
Accessing Captured Data
• As configured by GEC12 config, captured
data is at:
Host: ops.schooner.wail.wisc.edu
Username: gimsusr
Password: genigec12
Directory: /proj/GEC12/data/[exp ID]/
WireShark
• Packet analyzer
• Display packet in fields
• Selective view of packets by filters
Experimental Traffic
•
•
•
•
•
•
Packet generator: Iperf
Protocol: UDP
Source IP address: 10.1.1.3
Source port number: 48060
Destnation IP address: 10.1.1.2
Destination port number: 5001
Experimental Traffic
WireShark Usage
WireShark Usage
WireShark Usage
WireShark Usage
System Capabilities
•Current system is fairly low-end
•Dell 2650 PowerEdge server
•Onboard e1000 network card
System Capabilities
• In metadata file,
…
<stat_update timestamp="21:28:11">
<device_packets_received>11952058</device_packets_received>
<device_packets_dropped>23574</device_packets_dropped>
<device_interface_drops>0</device_interface_drops>
<packets_observed>11928476</packets_observed>
<bytes_observed>18035855712</bytes_observed>
</stat_update>
…
Stress Test – Capture Daemon
12
10
8
w/o Sampling
6
w/ Sampling
4
2
0
1000Mbps
100Mbps
1Mbps
1000Mbps w/o
1000Mbps w/
100Mbps w/o
100Mbps w/
1Mbps w/o
1Mbps w/
Samp.
Samp.
Samp.
Samp.
Samp.
Samp.
Pkt received
387508252
330336415
238551004
286693045
2241462
695764
Pkt dropped
44196832
27430032
542675
0
0
0
11.40539118
8.303665825
0.227488039
0
0
0
Ratio
** Sampled every 5th packet
Stress Test – NIC
90
80
70
60
50
w/o Sampling
40
w/ Sampling
30
20
10
0
1000Mbps
100Mbps
1Mbps
1000Mbps w/o
1000Mbps w/
100Mbps w/o
100Mbps w/
1Mbps w/o
1Mbps w/
Samp.
Samp.
Samp.
Samp.
Samp.
Samp.
Pkt received
387508252
330336415
238551004
286693045
2241462
695764
Pkt dropped
1538178305
1255481441
0
0
0
0
Ratio
79.87687817
79.16933438
0
0
0
0
** Sampled every 5th packet
Future Directions
• Wide variety of feature requests, bug fixes,
security improvements.
• Integration with other frameworks.
• Deployment of 5 more GIMS capture systems
across the US to make capture more widely
available.
Q&A