Towards Pervasive
Connectivity in Mobile
Computing
Frank Siegemund
European Microsoft Innovation Center
November 2006
1
Outline
 Motivation
 Related Work
 System Architecture
 Implementation
 Application
 Conclusions
2
Motivation
 Most middleware assumes basic connectivity
– Web Services
– Jini
– Corba
 Problematic in mobile scenarios
– Mobile P2P systems
– Mobile CSCW
 Heterogeneity
– Bluetooth, WLAN, IrDA, GPRS
– IPv4 vs. IPv6
– NATs and firewalls
3
Mobile P2P
 Bootstrapping problem
 Establish connectivity
between mobile users
 Mobile devices as firstclass citizens in P2P
applications
 Exploit knowledge
about connectivity in the
group
 More reliable/robust
group establishment
4
Scenario
 Content sharing
between mobile users
 Conference scenario
 Skiing trip
 Existing group collects
connectivity parameters
 Sharing by means of
visual tags/glyphs
 Handheld-embedded
cameras for capturing
tags
5
Outline
 Motivation
 Related Work
 System Architecture
 Implementation
 Application
 Conclusions
6
Related Work
 Capturing visual tags with handheld
devices and low-cost cameras
– Rekimoto et al.
 Visual tags in Pervasive Computing
– Rohs and Zweifel
– Many interesting scenarios
 Visual tags to bypass Bluetooth device
discovery
– Scott et al.
7
Related Approaches
 Out-of-band discovery for Bluetooth
– IrDA
– RFID
 Collaborative approaches for device discovery
– Siegemund and Rohs
 Manual configuration
 Send SMS with information about group
– Complicated from user perspective
8
Outline
 Motivation
 Related Work
 System Architecture
 Implementation
 Application
 Conclusions
9
System Architecture
10
Collecting Connectivity Data
 Overlay between group members
 Exploit heterogeneity
– Various communication technologies in group
– Different security constraints




Construct group connectivity profile
Create tags to transmit connectivity profile
Candidate group member captures tags
Simplified user interaction
11
Group Overlay
 Overlay can contain both mobile and stationary
nodes
 Overlay provides multicast group
 Nodes in group have synchronized state
 Connectivity properties of nodes shared between
group members
 NATs can be dealt with via different group peers
 Direct connections not always possible
 Group used to circumvent connectivity
constraints
12
Visual Tags
 Tags captured from video
stream
 Captured by handheldembedded digital
cameras
 Direct user feedback
about detected tags
possible
 Small video resolution
requires displaying
multiple tags
 Concrete tag layout
unimportant
13
Outline
 Motivation
 Related Work
 System Architecture
 Implementation
 Application
 Conclusions
14
Implemenation
 Heterogeneous environment
– Windows Mobile 5.0 smartphones
– Window XP/Vista desktops
 Implementation on desktops using
Windows P2P Infrastructure
 Mobile devices join/create P2P overlay
– Socket interface
– Lightweight database for data synchronization
 Native implementation
15
Visual Tags
 Tag detection in
DirectShow
 Analyze single video
frames
 Basic algorithm
Similar to
Rekimoto
et al.
– Black and white
conversion
– Region detection
– Identify guide bars
– Decode tags
– Determine tag index
– Finish if all tags have
been decoded
– Join group
16
Improvements
 Determine threshhold for black/white conversion
in feedback loop
– Different lightning conditions
 Mark tag in video frame to provide feedback to
users
– Tag detected or not
 Synchronization
– Tag contains time a single tag is displayed
– Capturing device can calculate schedule for reading
tags
17
Evaluation
 240 x 320 video frame resolution
 Implementation on 195 Mhz smartphone
– State of the art (no high-end model)
 400 ms per frame
– Can be improved significantly !!!
– Copy filter




90 ms for thresholding
250 ms for region detection
Algorithm deals with small rotations of the tag
More sophisticated tag detection systems
available
18
Outline
 Motivation
 Related Work
 System Architecture
 Implementation
 Application
 Conclusions
19
Example Application
 Mobile P2P chat
application
 Messaging for user
groups on mobile
devices
 Overlay to distribute
connectivity parameters
 Tags to exchange
connectivity profiles
 Deal with NATs and
connectivity constraints
20
Outline
 Motivation
 Related Work
 System Architecture
 Implementation
 Application
 Conclusions
21
Conclusions
 Basic connectivity problematic in mobile scenarios
 Current middleware solutions insufficiently address
this problem
 Facilitate groups to establish connectivity in mobile
scenarios
 Groups create finger print of network environment
 NATs and security constraints can be dealt with
 Visual tags user friendly way to communicate
connection properties
 Useful for mobile P2P and CSCW scenarios
 Real-life applications
22
Contact
Frank Siegemund
European Microsoft Innovation Center
Ritterstrasse 23, 52072 Aachen
franksie@microsoft.com
23