Infopipes
and the
Infopipe Stub
Generator
Galen Swint, Calton Pu,
Younggyun Koh, Wenchang Yan
Overview
 The Infosphere project
 Goals of the ISG
 Implementation
 Current results
 Infopipes and OEP
 Future Work
The Infosphere Project




Distributed computing with RPC is hard
RPC semantics do not fit streaming
applications
Need services to be more composable
QoS concerns
• “expanded” QoS
• eliminate redundant QoS coding

Enhance portability
• Same specification can be made for several
different comm. machines

RPC
Motivation
• Procedure call abstraction hides network



Application must address QoS
Does not understand application packets
Request/Response is an unnatural fit for
streaming media (movies/sound)
• Composable?
• Basis for CORBA, RMI, Sun RPC, SOAP

We need an information-centric
abstraction that is network aware to
complement RPC
Goals for ISL/ISG


Form the core of an Infopipe toolkit
ISL
•
•
•
•

Simple description for flows
Support datatypes
Support “basic” composition
QoS requirements
ISG
• Generate datatypes, communication stubs
• Support multiple communication layers
• Support multiple languages
Implementation







ISG becomes part of a process
XSLT + C++
Brand new XIP (Young)
New comm. added more quickly (but
needs more typing!)
XSLT allows easy modularity in
templates (xsl:include)
Must re-implement the aspects
Buzzword compliant
Implementation v0.2
μ-benchmarks

Round trip time (ping-pong)
• One number sent and received on separate
simplex channels
• 100 experiments of 1000 ping-pongs
Java RMI
ECho/Infopipe
TCP socket
Mean Time
Std. Dev.
1.5 sec
0.1 sec
0.35 sec
0.04 sec
0.152 sec
0.001 sec
Experiment 1 - UAV

UAV demo
• DoD project
• ECho
• Remote camera sends data via wireless
link
• Demonstrates code uploading
• ISG generates replacement comm. code
from SIP/XIP
Experiment 1 - UAV - Results

Initialization
• 101 runs, discard
first
• No statisical
difference

Original
Infopipe
Frame transfer
• 101 runs, discard
first
• No statistical
difference
Original
Infopipe
Mean Time
19.5 ms
19.6 ms
Std. Dev.
0.6 ms
0.7 ms
Mean Time
598 ms
606 ms
Std. Dev.
33 ms
45 ms
Same performance with 36% fewer LOC!
Conclusions





No performance hit
Substantial LOC savings
Time savings for development
Potential portability
LOC savings probably varies with App
OEP Integration Recap

Participation in BBN OEP
• Source-based filters for adaptive constrained
resource management in the Multi-UAV demo
• New filters as Infopipes (compression,
encryption)
• Better DRE application programs and better
QoS support (more platforms and dimensions)

Infopipe software tools
• Infopipe Stub Generator, program viewer,
other tools
• Demo apps (BBN OEP, Boeing OEP, etc)
New Infopipe Functionality

Three kinds of filters (and more)
• Image filters (lower/higher resolution, B&W)
• Compression (gzip, lzo, JPEG)
• Encryption (Rijndael)

Two underlying platforms
• BSD sockets/TCP and TAO AVStreams/UDP
• And more: event channels, publish/subscribe

Automated QoS management
• Adaptation by choosing appropriate
combination of filters (also controlled
manually at runtime)
Abstract Infopipe View
Defragmenter
Assembler
UDP
BBN Video
Distributor
DVDview
MPEG/PPM
ATR (PPM)
Wireless
Link-TCP
QuO
Contract
WebCam
Source
Filter
Filter
Control
GUI
Single Filter Experiments
Application
Infopipe Spec. (evolving)
Lang. (ISL)
GUI for Infopipe
Specification
XIP
ISL2XIP
(fixed,
extensible)
Infopipe Stub
Generator
XML
XML
parser
Stub
Middle
method
Stub
XML
generator
XML
Multiple-Filter Experiment (1)

Refining the information flow by
combining Infopipes through connectors
Source
Image Filter
Compress
Encrypt
DVDview
MPEG/PPM
Decrypt
Decompress
ATR
Multiple-Filter Experiment (2)

Self-configured Infopipes (when data
stream is unencrypted, system bypasses
the decryption stage)
Source
Image Filter
Compress
DVDview
MPEG/PPM
Decompress
ATR
Multiple-Filter Experiment (3)

Adaptive and flexible distribution of
Infopipe
Source stages at runtime
Intermediate Processing
UAV
Compress
Image Filter
Distributor
Decrypt
Encrypt
DVDview
MPEG/PPM
Decompress
ATR
Connector Specialization
Different machines
XML
Middle
XML
XML
Middle
XML
Socket Connector
Stub
Stub
Stub
Stub
parser
method
generator
parser
method
generator
Same machine
Same process/CPU
IPC Connector
Function Call Conn.
XML
Middle
Stub
Stub Function Call Conn. Stub Middle Stub XML
parser
method
method
generator
Future Work





Stabilize comm. layers
Some basic type checking
Use aspects for QoS (w/Lenin S.)
Wrap in specialization (w/Young K.)
Use aspects for performance
adaptation
Planned Experiments

Infopipe (filter) development and
integration
• Development cost/time, variety of filter code
• Code quantity (ISL/XIP module size), quality
(runtime overhead), portability (variety of
underlying platforms)
• Integration into OEP, making QoS work with
new filters (measure QoS dimensions below)

QoS dimensions and trade-offs
• Performance (e.g., latency, bandwidth, image
resolution)
• Security (e.g., encryption level)
• Other platforms in OEP that support QoS
Credits







Initial stub generator – Morimori
ECho – Greg Eisenhauer, Karsten Schwan
Java socket template – Younggyun Koh
C socket template – Volkan Altuntas
Initial QoS – Wei Han
Demo harnesses, demo integration –
Wenchang Yan
Initial UAV demo – Fabian Bustamante,
P@rick Widener
XIP Example

Creating a data type
• From the UAV demo, this holds a frame
<datatype name="Raw_data">
<arg type="integer" name="tag" />
<arg type="char" name="ppm1" />
<arg type="char" name="ppm2" />
<arg type="integer" name="size" />
<arg type="integer" name="width" />
<arg type="integer" name="height" />
<arg type="integer" name="maxval" />
<arrayArg type="byte" name="buff" size="155000" />
</datatype>
XIP Example

Creating a filter
• From the UAV example, this crops an
image to half size
<filter name="cropImage1Cto2C"
uri="cropImage.ecl"
inType="Raw_data1C"
outType="Raw_data2C"
takesParams="1"
paramType="Params_rectangle" />
XIP Example

Declaring a simple, singular pipe
• From UAV example, the sender and
receivers
• Note each is only a half-pipe
<pipe name="uavSender" absMachine="ECho2">
<outport name="send1" type="Raw_data" />
</pipe>
<pipe name="uavReceiverNormal" absMachine="ECho2">
<inport name="receive1" type="Raw_data" />
</pipe>
XIP Example

Connecting pipes together
<composedPipe name="multiUAV">
<declarations>
<pipe name="sender" class="uavSender" />
<pipe name="receiver" class="uavReceiverNormal" />
</declarations>
<ports />
<connections>
<connection>
<from pipe="sender" port="send1" />
<to pipe="receiver" port="receive1" />
</connection>
</connections>
</composedPipe>
ISG Template

Snippet for current version of ISG
• Submit function for sending data
• for-each command generates for multiple outputs
• “nodePipeOutConnect” captures some semantic
data about the template
<xsl:for-each
select="/xip/pipes/pipe[@name=$thisPipeClass]/outport">
<xsl:variable name="portName" select="@name"/>
<nodePipeOutConnect mark="true">
int <xsl:value-of
select="$thisPipeName"/>_<connType/>_<xsl:value-of
select="$portName"/><connName/>Submit( );
</nodePipeOutConnect>
</xsl:for-each>
Et voila!

The code produced
//int sender__send1Submit( CarrierStruct * );
int sender__send1Submit( );
μ-benchmarks

Single Integer
• Loop 100,000 times sending 1 integer (4 bytes)
ECho/Infopipe
TCP socket

Mean Time
2.6 sec
0.122 sec
Std. Dev.
0.015 sec
0.003 sec
1000 Integers
• Loop 10,000 times sending 1,000 integers
(4000 bytes)
ECho/Infopipe
TCP socket
Mean Time
3.463 sec
3.393 sec
Std. Dev.
0.004 sec
0.003 sec