Wireless Networks and Mobile Systems
Lecture 14
Pervasive Computing Applications
Lecture Objectives
● Understand characteristics and technical challenges
of pervasive computing applications
● Understand system middleware and support for
context-aware ubiquitous computing
● Exemplify pervasive computing applications
Pervasive Computing Applications
2
Sources
● T. Kindberg and A. Fox, “System software for ubiquitous
●
●
●
●
computing,” IEEE Pervasive Computing, Vol. 1, No. 1, 2002, pp.
70-81.
A. Smailagic and D. Kogan, “Location sensing and privacy in a
context-aware computing environment,” IEEE Wireless
Communications, Vol. 9, No. 5, Oct. 2002, pp. 10-17.
S.S. Yau, et al., “Reconfigurable context-sensitive middleware
for pervasive computing,’’ IEEE Pervasive Computing, Vol. 1,
No. 3, 2002, pp. 33-40.
M. Roman, et al., “A middleware infrastructure for active
spaces”, IEEE Pervasive Computing, Vol. 1, No. 4, 2003, pp. 7483.
B. Johanson, et al., “The interactive workspaces project:
experiences with ubiquitous computing rooms,” IEEE Pervasive
Computing, Vol. 1, No. 2, 2002, pp. 67-75.
Pervasive Computing Applications
3
Agenda
● Characteristics of pervasive computing
● System support to enable pervasive computing
● Middleware for context-aware pervasive computing
applications
● Examples of pervasive computing applications
■ Smart homes/rooms
○ UI’s Gaia-enabled Active Spaces
○ Stanford’s Interactive Workspaces project
● Experimentation
■ SLP-based pervasive Pocket TV application
Pervasive Computing Applications
4
Pervasive Computing Characteristics (1)
● Pervasive computing is:
■ An environment in which people interact with embedded
(and mostly invisible) computers and in which networked
devices are aware of their surroundings and peers and are
able to provide services or use services from peers
effectively
■ The creation of environments saturated with computing and
wireless communication, yet gracefully integrated with
human users
Pervasive Computing Applications
5
Pervasive Computing Characteristics (2)
● A pervasive (ubiquitous) system is characterized by:
■ Physical integration: integration between computing nodes
and the physical world, e.g., a whiteboard that records
what’s on
■ Instantaneous Interoperation: devices interoperate
spontaneously in changing environments, e.g., a device
changes its partners as it moves or as the context changes
Pervasive Computing Applications
6
Non-Examples of Pervasive Applications
● Accessing email over a phone line from a laptop
■ Neither physical integration nor spontaneous
interoperations
○ The laptop maintains the same association with a fixed
mail server
● A collection of wirelessly connected laptops based
on IEEE 802.11 at a conference
■ Discovery of the local network can be considered as
physical integration of laptops with a 802.11 access point
○ The physical integration in pervasive computing,
however, should involve a part of the environment that
has a non-electronic function (e.g., a white board, a cup,
etc.)
■ No spontaneous interoperation is possible without
considerable human manual intervention
Pervasive Computing Applications
7
Borderline Examples of Pervasive
Applications (1)
● A smart coffee cup and saucer
■ Physical integration: The cup is a regular
cup (a non-electronic function) that contains
sensing, processing and networking
elements that let it communicate with the
saucer of its state (full or empty, held or
putdown, hot or cold)
■ Instantaneous interoperation: Not satisfied
if a specialized protocol is used between the
cup and saucer as the owner would not be
able to use the coffee cup without the
saucer
Pervasive Computing Applications
8
Borderline Examples of Pervasive
Applications (2)
● P2P games
■ Physical integration: client devices can have
sensors that allow physical integration
■ Instantaneous interoperation: not satisfied if
preconfigured components are used in the
game
○ A more convincing example would involve
players with generic game playing pieces
that let them spontaneously join local
games not encountered before
Pervasive Computing Applications
9
Borderline Examples of Pervasive
Applications (3)
●
A Web-based object discovery system
■
Devices (e.g., laptops) have small, embedded web servers that
allow objects to be accessed by physical hyperlinks
■ The user is presented with a web page when it senses an
identifier of the object
●
Physical integration
■
●
smart devices can use web functionality to allow advertised
objects to be integrated into the physical world as web pages
without the need to reconfigure browsers
Instantaneous interoperation
■
not exactly satisfied since the system requires human
intervention to keep it going. The human in the loop changes
the browser’s associations to advertised web services
Pervasive Computing Applications
10
Examples of Pervasive Applications (1)
● A magic mirror that shows personal
data and actions of those users who
face it in a meeting
■ Physical integration: the mirror can sense
the presence of users in a meeting and
can record their actions, in addition to its
normal physical function
■ Instantaneous interoperation: the mirror
would interact with the room’s other
components the moment you switch it on
and would make spontaneous
association with all relevant local sources
of information about users
Pervasive Computing Applications
11
Examples of Pervasive Applications (2)
● A visitor brings his/her laptop into a
meeting room and without manually
configuring it in any way uses it to
send his presentation to the room’s
projector
■ Physical integration: the projector can be
activated from any laptop in the room
■ Instantaneous interoperation: A laptop can
spontaneously interact with the projector
and control the presentation with a proper
UI
■ Can be made context-sensitive, e.g.,
allowing only a particular visitor to do so
Pervasive Computing Applications
12
Other examples: Miró display
Cornell experiment: reflections on collective experience
Pervasive Computing Applications
13
Other examples: Gust of Presence
Delft Univ. of Technology: Gust of Presence project
Pervasive Computing Applications
14
The Semantic Rubicon
Image extracted from
IEEE Pervasive Computing
Magazine, Vol. 1, No. 1, 2002.
Little evidence exists
to suggest that software
alone can meet ubiquitous
computing challenges,
although it is desirable to
have minimum or no
intervention
The semantic Rubicon is the division between system and human
for high-level decision-making or physical world semantics processing
The division should be exposed in system design and the criteria and
mechanisms for crossing it should be clearly indicated
Pervasive Computing Applications
15
System Support
●
Discovery
■
Requiring a common “syntax and vocabulary” model for
specifying services
Service
Type
Mismatched
ServiceType=“photoPrint”
ServiceType=“printing”
Missed
opportunity
ServiceType=“digitalFrame”
Pervasive Computing Applications
16
System Support (2)
● Interaction
■ Requiring a “common” interoperation model for components
to interact
○ Event systems – publish, subscribe, and handle events
○ Tuple space – add, read, and remove tuples in a tuple
space
ServiceType=“Print”
DataType=“JPEG”
(Standardized)
Data Oriented
Interaction
Service
ServiceType=“print”
DataType=“JPEG”
ServiceType=“digitalFrame”
DataType = “JPEG”
Pervasive Computing Applications
A common service:
A tuple space or an event service
17
System Support (3)
● Adaptation
■ Without human intervention for achieving “calm computing”
as well as spontaneous interaction
■ Being able to display or manipulate data or UI from other
devices in a heterogeneous environment
■ Possible solutions include:
○ Dynamic downloading of mobile code to handle UI and
communication (as in Jini) – requiring JVM
○ Dynamic UI generation/transformation (as in Stanford’s
Interactive Workspaces) – requiring a common UI
description language to describe UI elements, e.g., “This
control is a Boolean toggle”
 Another example is to use XML for specifying UI for
web services
Pervasive Computing Applications
18
Human Interface Adaptation
● Use the same high-level UI description language to
generate device-specific UIs to control lights and
displays in a smart room for (a) a desktop HTML
browser, (b) a Java-equipped device, and (c) a Palm
device
Pervasive Computing Applications
Image extracted from
IEEE Pervasive Computing
Magazine, Vol. 1, No. 1, 2002.
19
System Support (4)
●
Physical Integration
■
●
Context-awareness support is
required to allow an application to
access different types of contexts
(e.g., locations, identities, QoS
conditions) while hiding how the
information was sensed or collected
Examples:
■
■
“Show me the agenda!” - can be
made context-sensitive
“Show me related Web pages for
museum exhibits!” – can be based on
the location and ids of exhibits
sensed through barcode or Infrared
beacon
Context aware
I know
I know
who I am
you provide
photographing photo printing
services
tag
Pervasive Computing Applications
tag
20
Positioning for Location-aware
Computing
● Outdoor – GPS (Global Position Sensing) normally to
within 1-5 m accuracy with differential correction
● Indoor
■ Active Bat (shown below) developed by AT&T Lab
Cambridge based on ultrasonic sensors (to within 3 cm
accuracy)
■ CMU-TMI (shown next) developed by CMU based on 802.11
for location tracking
(to within 1-5 m accuracy)
Image extracted from
IEEE Pervasive Computing
Magazine, Vol. 1, No. 1, 2002.
Pervasive Computing Applications
21
System Support – Location Tracking in
Smart Spaces
●
CMU has developed and
deployed a Triangulation,
Mapping and Interpolation (CMUTMI) algorithm based on 802.11
for location tracking:
■
■
■
■
Physical locations of access
points are known in advance
A mobile device scans all access
points (at least 3) within range to
determine their signal and noise
strengths
The signal strengths are used to
infer the distance between the
device and the access points
using an approximate relation
obtained a priori
Triangulation technique is
applied to calculate the physical
location
Pervasive Computing Applications
22
System Support (5)
● Programming Framework
■ A programming framework for pervasive computing can be
placed at the application or middleware level:
○ Application level
 Providing only application-level coordination
mechanisms, e.g., through a “tuple-space” API
○ Middleware level
 Providing more tightly integrated facilities for
achieving context awareness, QoS resource
management, and adaptive control.
 Determining the user’s task and intention, and
facilitating associations between components to
assist the user in these activities
Pervasive Computing Applications
23
System Support (6)
● Robustness
■ Failure is a common case in mobile and pervasive
computing
■ Recovering from failures  always prepared to reacquire
lost resources
○ Expiration-based schemes and soft states
 Periodic service advertisement and lease in the
“common” data-oriented interaction service
○ Separating failure-free (logic) from failure-prone (e.g.,
accessing a service or file while moving) application code
 More effective failure handling can be done implicitly
by the middleware, e.g., retrying an idempotent
operation (binding to a remote file) for a number of
times
● Security – trust between devices, access control
Pervasive Computing Applications
24
Middleware for Pervasive Computing
● Context-Aware middleware for pervasive computing
must address two “broad” characteristics:
■ Tradeoff between awareness and transparency
○ For pervasive applications, environment awareness is
key to their effectiveness
■ Cooperation between development support and run time
services
○ Let developers focus on application logic
Pervasive Computing Applications
25
RCSM: A Reconfigurable Context
Sensitive Middleware
● ASU’s RCSM addresses the following “specific” characteristics:
■ Uniform development support
○ providing a uniform way to express context awareness without
restricting to a specific language, OS, or environment
■ Application-specific context acquisition, analysis and detection
○ allowing developers to express the need for context data in a
platform-independent way without knowing “how” data are
obtained
■ Context-triggered actions
○ transparently invoking actions whenever the specified contexts
are valid
■ Transparent support for ad hoc communication
○ abstracting the details of ad hoc communication from
applications including proactively discovering new devices,
establishing new communication links and notifying the
application when a device is found
Pervasive Computing Applications
26
RCSM: A Reconfigurable Context
Sensitive Middleware (2)
●
ASU’s RCSM provides a
context-aware interface
description language
(CA-IDL) API for an
application to
■
■
●
Define contexts used by
the application
Specify actions taken
based on context values
Compilers are provided
to build adaptive object
containers (ADCs) based
on CA-IDL
Pervasive Computing Applications
Image extracted from
IEEE Pervasive Computing
Magazine, Vol. 1, No. 3, 2002.
27
Pervasive Computing Example 1:
UI’s Gaia-enabled Active Spaces (1)
● Gaia is a distributed middleware
infrastructure that coordinates
software entities and
heterogeneous network devices
contained in a physical space
■ The physical space is extended into
an “active space” by adding
coordination via a context-based
software infrastructure
Image extracted from
IEEE Pervasive Computing
Magazine, Vol. 1, No. 4, 2002.
Pervasive Computing Applications
28
Pervasive Computing Example 1:
UI’s Gaia-enabled Active Spaces (2)
●
●
●
●
●
Event manager service: for event channel registration/notification
Context service: for querying/registering context information such as people’s
locations
Presence service: for maintaining soft states of resources including digital
entities (applications and services using heartbeats) and physical entities
(devices and people using sensors)
Space repository: for displaying/retrieving information about all entities in the
active space
Context file system: associating data with context and format info to allow
context-sensitive data retrieval presented with the right UI format, e.g., make
personal data available to applications conditioned on presence
Example: Presentation
Manager Application
Image extracted from
IEEE Pervasive Computing
Magazine, Vol. 1, No. 4, 2002.
Pervasive Computing Applications
29
Pervasive Computing Example 1:
UI’s Gaia-enabled Active Spaces (3)
● EX: Presentation Manager
Application
● Speaker X with an RF active badge
and a handheld walks into a Gaiaenabled room
● Presence service detects the badge
and sends a “X is here” event with
the user profile info of X contained
■ Space repository receives the event
and displays the user info.
■ Context file system obtains the
speaker’s mount point and mounts
the speaker’s presentation file ready
for access
● The handheld (controller) detects
the directory server and then sends
a heartbeat event to the heartbeat
channel
■ Presence service detects and sends a
“new device” found event
■ Space repository receives the event
and displays the handheld info
● Both speaker X and handheld are
now entities of the active space
Pervasive Computing Applications
A prototype of Gaia-enabled active space
Context: Speaker X enters the room
Action: start the presentation manager
application
• Read the presentation file
• Allowing X to control presentation
by control events generated from the
touch screen or handheld
30
Pervasive Computing Example 2:
Stanford’s Interactive Workspaces (1)
● Stanford’s Interactive Workspaces is based on a
tuple space data model
■ Event Heap: Allowing name-type-value events (with
expiration times) to be posted and retrieved
■ Data Heap: Allowing data with attributes (e.g., format) posted
■ iCrafter: Providing service advertisement/invocation and a UI
generator that returns the best interface for the device
Image extracted from
IEEE Pervasive Computing
Magazine, Vol. 1, No. 2, 2002.
Pervasive Computing Applications
31
Pervasive Computing Example 2:
Stanford’s Interactive Workspaces (2)
● Room-based cross-platform interfaces
■ The room-control system stores the geometric arrangement
of screens and lights in the room in a configuration file and
will automatically provide controllers on any device
supporting a UI format available through iCrafter
html
Java Swing
Palm
Pervasive Computing Applications
Image extracted from
IEEE Pervasive Computing
Magazine, Vol. 1, No. 2, 2002.
32
Experimentation:
SLP-based Pervasive Pocket TV
● SLP (RFC 2608) is an IETF standard that provides a scalable
framework for automatic resource discovery on IP networks.
Three entities are defined in a SLP system:
■ User agent (UA)
■ Server agent (SA)
■ Directory agent (DA)
● A UA initiates service discovery on behalf of one or more
applications. It can send queries to all SAs via multicast if a DA
does not exist or to a DA to discover services via unicast.
● A SA works on behalf of one or more services. It responds
directly to UA queries via unicast. If a DA exists, a SA can
register its services with the DA to expose its services.
Pervasive Computing Applications
33
Experimentation:
Pervasive Pocket TV (2)
● A DA serves as a centralized information repository.
It accepts SA registrations and answers UA directory
service queries. The DA support is optional and is
introduced for performance and scalability
considerations.
Application
Service
query
UA
SA
reply
directory service query
service registration
DA
directory service reply
Pervasive Computing Applications
acknowledgment
34
Experimentation:
Pervasive Pocket TV (3)
●
DA Discovery: Passive versus Active
Multicast SrvRqst (service:directory-agent)
UA/SA
DA
Unicast DAAdvert
Multicast DAAdvert
UA/SA
Pervasive Computing Applications
DA
35
Experimentation:
Pervasive Pocket TV (4)
●
Three configurations
Small: No DA
SA
Multicast SrvRqst
Multicast SrvRqst
UA
Unicast SrvRply
SA
Unicast SrvRply
Unicast SrvReg
Medium: 1 DA
SA
Unicast SrvAck
Unicast SrvRqst
UA
DA
Unicast SrvRply
Unicast SrvAck
SA
Unicast SrvReg
Large: Multiple DA’s
SA
SA
UA
DA
DA
UA
SA
Pervasive Computing Applications
SA
36