Scalable Coordination Algorithms
for Deeply Distributed Systems
PIs:
Deborah Estrin (UCLA and USC-ISI)
John Heidemann (USC-ISI)
Ramesh Govindan (USC-ISI)
http://www.isi.edu/scadds
Technical staff:
Fabio Silva (USC-ISI)
Students (SCADDS USC-ISI, and UCLA):
Alberto Cerpa, Jeremy Elson, Deepak Ganesan, Lewis
Girod, Chalermak Intanagowat, Ya Xu, Yan Yu, Jerry
Zhao
7/24/2016
1
Outline
• Diffusion
– Testbed measurements (Silva, Intanago)
– In network processing:
• Nested Queries (Silva, Intanago)
• Aggregation (Intanago)
• Tracking (Ganesan, Work in progress)
– Scaling mechanisms
• GEAR (Yu) and GAF (Xu) routing
• TinyDiffusion (Ganesan)
• Tiered testbed update
– PC-104+, UCB Motes with TinyOS, Tags
– MAC (Ye)
• Plans for Q2-3 ’01
7/24/2016
2
Experiments on our PC104
testbed
• Initial experimental measurements of
diffusion (e.g., for comparison with
simulation)
– Compare bytes sent by diffusion with and without
aggregation (simple in network processing)
• Measurement Setup
– A 5-hop network of 14 nodes on 2 ISI floors
(testbed is actually 30 nodes and growing)
– Radio: 13kbps radiometrix
– 1 sink and 1-4 sources (each source sends 112
bytes every 6 seconds)
7/24/2016
3
Experimental Results
• Bytes sent by diffusion per event vs. Number of sources
Diffusion without suppression
Diffusion with suppression
7/24/2016
4
Comparison to Simulation
• Bytes sent by diffusion per event vs. Number of sources
Diffusion without suppression
Diffusion with suppression
7/24/2016
5
Differences between
Simulations and Experiments
• MAC differences
– Modified 802.11 for simulations to represent
hybrid TDMA-Contention
– Radiometrix MAC for experiments
• Channel differences
– No obstacles used in ns-2 simulations
• Note: we have added ability to include simple “terrain”
but didn’t try to replicate indoor exp terrain in sims
– More packet losses and collisions in experiments
• Collisions in experiments act as unintentional suppression
(make no suppression look better than it will with better
mac)
7/24/2016
6
In network processing:
Nested Queries
• Edge processing overwhelms power and bandwidth
consumption
• Nested queries where low-energy sensors trigger
high-energy sensors
Edge Processing
Nested Queries with In-network Processing
7/24/2016
7
Experimental Validation:
Testbed Measurements
• Higher delivery ratio for nested query indicates that
localizing data traffic benefits performance.
• % Audio Events Successfully Delivered vs. Number of
light sensors
Nested query
1-level query
7/24/2016
8
Reinforced Aggregation
• Promote In-network Data Aggregation near
the Sources for Better Energy Savings
• Two Approaches for Reinforced Aggregation
– Greedy Tree Approach
• Incremental approach -- Adds minimum number of links
on the existing tree
– Iterative Approach
• Selects aggregation points such that energy dissipation
for delivering aggregated data is approximately
minimized
7/24/2016
9
Greedy Tree Approach
(incremental)
Source 2
A
C
Source 1
B
D
d2*:1:B
d2:1:A
d1:0:B
d2*:1:B
d2:1:A
d1:1:B
d2*:1:D
E
Sink d2:2:C
d1:2:D
7/24/2016
• Each node enumerates additional
cost for supplying additional data
samples of the same data type for
previously reinforced path (ontree)
– On-tree nodes don’t increment cost
for additional data samples
• Sink node selects path for
particular data samples based on
cost advertised on the existing
tree, and that advertised on other
(possibly shorter) paths
– Advertised cost along existing tree
reflects sharing
• Each node maintains message cache
[message:energy:last hop]
10
Iterative Approach
Source 2
A
C
E
Sink
7/24/2016
• Each node advertises cost
for each data sample
• Each node also advertises
Source 1
cost for each aggregate
d2:1:A
(multiple data samples
B
belonging to same data type)
d1:0:B
d1&2:1:B • Sink reinforces aggregate
with minimum advertised
D d2:1:A
energy cost
d1:1:B
• Each node maintains message
d1&2:2:D
cache [message:energy:last hop]
d2:2:C
d1:2:D
d1&2:3:D
11
Planned: Tracking-based in
network processing
• Work in progress on other “primitives” such
as tracking (example motivated by Xerox and
U Wisc)
• Edge processing:
– Node A with detection subscribes to other nodes
that it (A) believes might “see” tracked object and
contribute most to location/tracking
• In network processing:
– Node A with detection sends out interest
containing attributes and function that
characterizes locations/nodes that might “see”
tracked object and contribute most to
location/tracking
7/24/2016
12
Scaling Mechanisms
• Flooding of interests:
– Geographic and Energy informed routing of
interest messages
• Exploiting redundancy:
– Geographic Adaptive fidelity applied to topology
used for flooding interests
• Optimizations for large numbers of listeners:
– Pushed data (e.g., needed by Univ Wisc API)
discussed in Integration meeting (see John H.
slides)
• Optimizations for much smaller/constrained
nodes
7/24/2016
13
Geographical and Energy Aware
Routing (GEAR)
• Motivation:
– Reduce overhead of interest and low rate data
flooding in directed diffusion
• Basic ideas:
– Leverage geographical information to restrict
flooding, and recursively disseminate data inside the
target region.
– Extend overall network lifetime using local
techniques to balance energy usage
– Reuse routing information across multiple user
queries.
– Extension of GPSR, LAR, other geographic routing
7/24/2016
14
GEAR
• Forward the packets towards the target
region:
– Greedy mode
• minimizing cost function (f=mix function of distance and
energy)
– Route around “communication holes” with energy
aware neighbor estimation
• Disseminate the packet within the target
region:
– Geographic Recursive Forwarding
• recursively re-send packets to sub-regions of the original
geographic region
– Restricted Flooding
• apply in low density case.
7/24/2016
15
Simulation results: multiple traffic pairs
# packets delivered before network partition vs. # nodes
GEAR
GEAR Geo-only
7/24/2016
16
Simulation results: multiple traffic pairs
# connected pairs broken down per received data
packet vs. # nodes
GEAR
GEAR Geo-only
7/24/2016
17
GEAR Plans
• Prototype Implementation on our testbed in
progress (Yu)
• Planned experimentation w/CSIP support
– Desired data is characterized by geographic
attributes
– Xerox and U. Wisc as users/collaborators
• Planned addition of data-dissemination-cost
attribute
– Support CSIP “informed” decision re. data
contribution (to task) vs. dissemination cost
7/24/2016
18
TinyDiffusion
• Implementation of Diffusion on
resource constrained UCB motes
– 8bit CPU, 8K program memory, 512 bytes data
memory
• Subset of full system
– retains only gradients, and condenses attributes
to a single tag.
• Entire System runs for less than 5.5 KB
memory
– TinyOS adds ~3.5K and 144 bytes of data. (incl.
support for Radio and Photo Sensor)
– Diffusion adds ~2K code and 110 bytes of data to
TinyOS.
7/24/2016
19
TinyDiffusion Functionality
• Resource Constraints
– Limited cache size: currently 10 entries of 2bytes each
– Limited ability to support multiple traffic streams.
Currently supports 5 concurrently active gradients.
• Tiered Deployment
– PC104s running diffusion interface with mote clusters
using TinyDiffusion.
– Motes enable dense sensor deployment but can support
limited in-network processing
– Logical Header format of TinyDiffusion is compatible
with the Diffusion header.
7/24/2016
20
Gateway Architecture
Photo
Data Source
Data Sink
TinyDiffusion
TINYOS
Acoustic
Data Source
Mote-NIC
MOTE
Query
Data Sink
RFM
DIFFUSION
LINUX
7/24/2016
Device
Driver
MOTE
ATMEL 8586 4MHz
MCU
8K program memory
512 Bytes PC104
Data
Memory
RFM Radio 900 MHz
Transceiver
Serial
TINYOS
PC104
AMD Elan™SC400
66MHz CPU
16MB RAM
Form Factor: 3.6" x 3.8" x 0.6"
21
Tiered Testbed
•
•
•
•
PC-104+(linux) with MoteNIC
Tags, Sensor Card
UCB Motes w/TinyOS
Yet to come: SmartDust (highly specialized
nodes)
PC/104
Tag
7/24/2016
UCB Mote
22
“Shoebox Testbed v2”
Featuring:
• PC-104+ w/
Pentium 266
• Mote-NIC
• Ethernet for
debugging and
measurement
• Linux 2.4.2
w/glibc 2.1.3
• Plastic
shoeboxes
from local
drugstore
7/24/2016
23
Sensor Card
• The sensor card is a small (2”x4”)
microcontroller board with several
on-board sensors and emitters
– Microphone
– Light sensor
– Accelerometer
• Designed to perform simple sensing tasks at low
power.
– Currently it is connected to the PC-104 platform by serial.
– Data is preprocessed on the sensor board and fed back to
the PC-104 for analysis and communication.
– The next version of the PC-104 platform will have the
capability to be awakened by a peripheral such as the sensor
card.
7/24/2016
24
Plans Q2-4 ‘01
• Diffusion Experimentation:
– larger scale experiments and tuning
– port to WINSng 2.0 platform
– TinyDiffusion experiments and interoperate with Diffusion
• In Network Processing
– Develop primitives for tracking
– Implement in network aggregation
• Scaling enhancements
–
–
–
–
Geographic/Energy Adaptive Routing in Diffusion 3
Adaptive fidelity experiments (applied to interest flooding)
Data Push (Univ. Wisc. API)
Bulk transfer capability (e.g. for mobile code, larger sensor
data)
• SenseIT experimentation support
– November Demo participation
– Emulation of diffusion over wired networks for debugging
7/24/2016
25
Related (other funding) Projects
• Active cooperative localization
– for sensor network self configuration when no/subset GPS
• ASCENT
– Self-configuring topology for densely deployed networks
• Adaptive beacon placement/activation
– For proximity based localization
• Computation primitives and constructs
– Beyond nested queries (w/ Culler)
• Application projects:
– Habitat monitoring (Biocomplexity mapping)
– Ecophysiology (w/ Culler, Pister, Rundel)
7/24/2016
26
Publications/Submissions
http://www.isi.edu/scadds
• Mobicom submissions (adaptive fidelity,
multipath, adaptive beacon placement)
• SOSP submission (naming-based architecture)
• IROS (Robotics) submission (localization)
• ICDCS (address-free, beacon placement)
• IPDPS (time synch)
• Sigcomm submission (self-config topology
experiments)
7/24/2016
27