Capturing an Experiment: A
Wireless Testbed Perspective
Ivan Seskar
Seskar (at) winlab (dot) rutgers (dot) edu
Rutgers, The State University of New Jersey
www.winlab.rutgers.edu
WINLAB
Deploy & Configure
Experiment Support Architecture
Control
Definition
System
Experimenter
Measure
Results
WINLAB
Orbit Services
ORBIT Interface
Chassis Manager Controller
Experiment Management
Service
Orbit Nodes
Collection Servicee
Experiment Controller
Chasis Manager (CM)
Collections Manager
TFTP Server
Resource Controller
Collection Server
Frisbee Server
*
Orbit Measurement Lib
-End2
berkeley DB
-End1
APT Server
*
SQL DB
WINLAB
Orbit Measurement Library (OML)
User application
Orbit node
Collection server
OML data filter, id = xx
OML data filter, id = yy
OML data filter, id = zz
pluggable filters, chosen
by the experimenter
OML interface to user application
OML SQL module
SQL DB
OML XDR decoder
Berkeley DB
OML transport layer
OML transport layer
XDR Encoded data over multicast channel.
WINLAB
Defining an Experiment
Properties
App 1
Test code
Sys
Measurements
Prototype
Application
Topology
P1
Mapping Strategy
- Mapping
nodes to prototype
Pi
- Binding
free properties
Dynamic
Static
Experiment
WINLAB
Experiment Definition
OMF Experiment Description Language, OEDL
Experiment.name = "tutorial-1"
Experiment.project = "Orbit::Tutorial"
Experiment.startMode = Experiment::REBOOT
#
# Define nodes used in experiment
#
node([2, 3], :sender) {|n|
n.prototype("http://apps.orbit-lab.org/sender", {
:if => Node::W0_IF,
:packetSize =>128
Experiment.parameter("packetSize"),
:rate => Experiment.parameter("rate")
})
}
node([3, 4]) {|n|
n.prototype("http://apps.orbit-lab.org/receiver",
{:if => Node::W0_IF})
}
#
# Configure environment
#
node("/*/*").net.w0 {|n|
n.essid = "HelloWorld"
n.mode = "ad-hoc"
n.channel = 1
n.xmitPower = 1
n.bitrate = "11Mbps" # Force single rate
n.ip = "%10.0.%x.%y"
}
# Start experiment
whenReady {|e, n|
p.packetSize = 1024
p.rate = 250
e.sleep 5.0
e.done
}
WINLAB
ORBIT Grid
WINLAB
Issues Affecting Archiving/Repeatability
Hardware
 Differences in commercial hardware
 Difference in tolerances and ageing for various low
cost components
Software
 Software or firmware bugs that may manifest as
inconsistent experimental results
Environment
 Uncontrolled interference over time and space
 Co-located infrastructure, opening and closing of
doors, movement of people etc.
8
WINLAB
Device Calibration Setup
9
WINLAB
Calibration: Device Characterization
• Calibrate transmit and receive side of the wireless cards
• Record corrections to be applied for each card based on
the discrepancies observed
Transmitter calibration
Reciver calibration
10
WINLAB
Calibration: Temporal repeatability
The Experiment
11
WINLAB
Calibration: Spatial repeatability
Blue sender sends to blue receiver
Yellow sender sends to yellow receiver
Experiment repeated on a different set
of nodes with same topology
Spatial throughput variations
w.r.t mean
for experiment
duration averaged over
different experimental runs
12
WINLAB
Calibration
• Node calibration procedure necessary to
address hardware dependencies
• Temporal variation less than device variability
• Periodic re-calibration necessary
• Need to automate the on-site calibration
procedure so that it can “monitor” the testbed
“drift”
• Bounds on acceptable performance
– Baseline standard
13
WINLAB
Archiving ORBIT Experiments
Existing Support:
•
•
•
•
•
Image
Experiment script and/or commands on the console
State of services and environmental traces
Measurements (if using OML)
Device calibration info
What is missing
• Full metadata
• State of devices
• Full RF environment (not necessarily critical for repeatability)
WINLAB
Wild Environment: WiMAX Basestation
Basestation
(IDU) Unit
RF (ODU)
Amplifier
Roof
Antenna
• Wide-area packet based communication system (4G)
• Operational with an educational license
WINLAB
Wild Environment: Coverage
Measured
RSSI
WINLAB
Wild Environment: Mobility
Trajectory
RSSI
WINLAB
Instead of Conclusion
• Repeatability/replayability
incentive
• Standardized anomaly monitoring
• Storing metadata in archives
• Baseline performance tests and
calibration
WINLAB