Advanced media-oriented systems
research: Ubiquitous capture, access, and
interpretation

Faculty involved with RI-related projects


Federal funding


NSF RI, NSF ITR, DARPA, DOE
State funding


Kishore Ramachandran, Mustaque Ahamad, Karsten
Schwan, Richard Fujimoto, Ken Mackenzie, Sudha
Yalamanchili, Irfan Essa, Jim Rehg, Gregory Abowd,
Yannis Smaragdakis, Santosh Pande, Calton Pu, Ling Liu, ...
Yamacraw, GT Broadband Institute
Industry funding (equipment and personnel)

HP, Intel, Microsoft, IBM
MediaBroker and DFuse:
Powerful support for emerging
applications
Kishore Ramachandran
Jim Rehg, Phil Hutto, Ken Mackenzie, Irfan Essa
College of Computing, Georgia Tech
Kath Knobe, Jamey Hicks, Raj Kumar
HP Labs
Students:
Rajnish Kumar, Matt Wolenetz, Ilya Bagrak, Martin Modahl,
Bikash Agarwalla, Junsuk Shin, Arnab Paul, Sameer Adhikari,
Nissim Harel, Hasnain Mandviwala, Yavor Angelov, David Hilley
www.amonline.net.au/invertebrates/mal/gallery/octopus.htm

Distributed system components

sensors, actuators, embedded processors, PDAs,
laptops, clusters…
camera
Skiff
camera
Skiff
Sensors
Actuators
Sensor
Fusion
Data Aggregators
Unix / Linux / NT cluster
“OCTOPUS” DIAGRAM
head / arms / tentacles
Killer App?

Application context



distributed sensors with varying capabilities
control loop involving sensors, actuators
rapid response time at computational
perception speeds
Emerging Applications


Distributed Collaboration
Emergency Response



Collaborative search and rescue
Evacuation management
“Aware” Environments
Key Requirements
 Data distribution and Infrastructure
adaptation
Application Characteristics







Physically distributed heterogeneous devices
Interfacing and integrating with the physical environment
Diverse stream types (low to high BW)
Diverse computation, communication and power capabilities
(from embedded sensors to clusters)
Stream fusion/transformation, with loadable code
Resource scarcities
Dynamic join/leave of application components
Two projects:
 MediaBroker
 DFuse
Abstractions…abstractions

Goals:



Temporal guarantees, efficient streams,
distributed synchronization, fusion support
Simplify application development
Key result: Fusion Channels

built on top of D-Stampede [ICDCS 2002]
A Fusion Channel (a ‘Virtual Sensor’)
Inputs
F()
...
...
(sensors or
other fusion
channels)
Consumers
(actuators or
other fusion
channels)
The MediaBroker

An architecture for Data Distribution


Type management, Scalability, Stream “sharing”
A sample scenario for distributed collaboration



Physically distributed participants in a classroom setting
Interaction via a variety of devices
System takes care of any needed data transformations and
distribution
MediaBroker
Infrastructure
MediaBroker Infrastructure

(
)





audio
video
sonar

Fusion Channels and Virtual
Sensors
Resource Discovery and Sharing
Sensor-Provided Attributes
Sensor-Derived Attributes
Scalability
Access Control and Resource
Scheduling
Security
bio
derivation
fusion
sharing
registry
Re-publish
discovery
D-Stampede
virtual
sensors
applications
MediaBroker and Typed Streams

Experimental “type-lattice” registry, discovery, and
resolution structure
Fusion Channel
Performs text-to-speech
Fusion Channel
Outputs A/V as MPEG,
Incorporates
text as caption
Today’s
Lectur
e:
DFuse


[ACM SenSys 2003]
An architecture for Infrastructure Adaptation
A sample scenario for an aware environment




Field trip for a class!
Deployed power-constrained sensors
Dynamic wireless network consisting of the students’
PDAs
In-network stream filtering and aggregation
DFuse Fundamentals

Sensors
Fusion Module: Deploys
task graph on sensor
network

Collage

Sink (Display)
Filter
Cameras

Placement Module:
Employs a self-stabilizing
algorithm to place fusion
points in the network

Sample surveillance application task graph:
filter and collage are the fusion functions.


Deployed on iPAQ farm!
Tested 4 cost-functions
Comprehensive API for
fusion and migration
Low-overhead

Energy and application
aware cost functions
Localized decisions
Evaluation
Application Timeline showing Network Traffic
Applause!!!
