Pervasive Computing: Vision and
Challenges
M. Satyanarayanan
IEEE Personal Communications, 2001
Evolution & Related Fields
• Pervasive computing represents a major
evolutionary step in a line of work dating back
to the mid-1970s.
• Two distinct earlier steps in this evolution:
– Distributed systems
– Mobile computing
Evolution & Related Fields
Figure 2. Taxonomy of computer systems research problems in pervasive computing.
Evolution & Related Fields
• Distributed systems
– Arose at the intersection of personal computers and local
area networks.
– The research that followed from the mid-1970s through the
early 1990s created a conceptual framework and algorithmic
base (for various types of networks)
– Spans many areas that are foundational to pervasive
computing
Evolution & Related Fields
• Mobile computing
– Led by the appearance of full-function laptop computers and
wireless LANs in the early 1990s
– How to build a distributed system with mobile clients??
– Many basic principles of distributed system design continued
to apply
– Four key constraints of mobility forced the development of
specialized techniques:
• Unpredictable variation in network quality
• Lowered trust and robustness of mobile elements
• Limitations on local resources imposed by weight and size
constraints
• Concern for battery power consumption
Other related fields
• Sensor networks
• Human-computer interaction
• Artificial intelligence
Other related fields
• Sensor Networks
– A sensor network consist of a large
number of tiny autonomous
computing devices, each equipped
with sensors, a wireless radio, a
processor, and a power source.
– Sensor networks are envisioned to
be deployed unobtrusively in the
physical environment to monitor a
wide range of environmental
phenomena
• e.g., environmental pollutions, seismic
activity, wildlife
Other related fields
• Human Computer Interaction (HCI)
– HCI is the study of interaction between people
(users) and computers.
– Goal of HCI: to improve the interaction between
users and computers by making computers more userfriendly and receptive to the user's needs.
– Long term goal of HCI: to design systems that
minimize the barrier between the human's cognitive
model of what they want to accomplish and the
computer's understanding of the user's task.
Other related fields
• Artificial Intelligence
– AI can be defined as intelligence exhibited by an
artificial (non-natural, manufactured) entity.
– AI is studied in overlapping fields of computer
science, psychology and engineering, dealing with
intelligent behavior, learning and adaptation in
machines, generally assumed to be computers.
– Research in AI is concerned with producing machines
to automate tasks requiring intelligent behavior.
Key research thrusts
• Pervasive computing incorporates four additional
research thrusts:
–
–
–
–
Effective use of smart spaces
Invisibility
Localized scalability
Masking uneven conditioning
Effective use of smart spaces
• By embedding computing infrastructure in building
infrastructure, a smart space brings together two
worlds that have been disjoint until now.
• The fusion of these worlds enables sensing and
control of one world by the other.
– Automatic adjustment of heating, cooling, and lighting levels
in a room based on an occupant’s electronic profile.
Invisibility
• The ideal expressed by Weiser is complete
disappearance of pervasive computing technology
from a user’s consciousness (minimal user distraction).
• If a pervasive computing environment continuously
meets user expectations and rarely presents him with
surprises, it allows him to interact almost at a
subconscious level.
Localized scalability
• As smart spaces grow in sophistication, the intensity of
interactions between a user’s personal computing space
and his/her surroundings increases.
• This has severe bandwidth, energy, and distraction
implications for a wireless mobile user.
• The presence of multiple users will further complicate this
problem.
• Good system design has to achieve scalability by severely
reducing interactions between distant entities.
Masking uneven conditioning
• Huge differences in the “smartness” of different
environments — what is available in a well-equipped
conference room, office, or classroom may be more
sophisticated than in other locations.
• This large dynamic range of “smartness” can be
jarring to a user, detracting from the goal of making
pervasive computing technology invisible.
• One way to reduce the amount of variation seen by a
user is to have his/her personal computing space
compensate for “dumb” environments.
Detailed design and implementation issues
•
•
•
•
•
•
•
•
•
User intent
Cyber foraging
Adaptation strategy
High-level energy management
Client thickness
Context awareness
Balancing proactivity and transparency
Impact on layering
Privacy and trust
User intent
• For proactivity to be effective, it is crucial that a
pervasive computing system track user intent.
– Otherwise, it will be almost impossible to determine which
system actions will help rather than hinder the user.
• For example, suppose a user is viewing video over a
network connection whose bandwidth suddenly drops.
Should the system:
– Reduce the fidelity of the video?
– Pause briefly to find another higher-bandwidth connection?
– Advise the user that the task can no longer be accomplished?
• The correct choice will depend on what the user is
trying to accomplish.
Cyber foraging (aka “living off the land”)
• Dynamically augment the computing resources of a
wireless mobile computer by exploiting wired hardware
infrastructure.
• As computing becomes cheaper and more plentiful, it
makes economic sense to “waste” computing resources
to improve user experience.
• In the forseeable future, public spaces such as airport
lounges and coffee shops will be equipped with compute
servers or data staging servers for the benefit of
customers, much as table lamps are today.
– Today, many shopping centers and cafeterias offer their
customers free wireless internet access.
Adaptation strategy
• Adaptation is necessary when there is a significant mismatch
between the supply and demand of a resource (e.g. wireless
network bandwidth, energy, computing cycles or memory).
• There are three alternative strategies for adaptation in
pervasive computing:
– A client can guide applications in changing their behavior so that
they use less of a scarce resource. This change usually reduces the
user-perceived quality, or fidelity, of an application.
– A client can ask the environment to guarantee a certain level of a
resource (reservation-based QoS systems). From the viewpoint of
the client, this effectively increases the supply of a scarce
resource to meet the client’s demand.
– A client can suggest a corrective action to the user. If the user
acts on this suggestion, it is likely (but not certain) that resource
supply will become adequate to meet demand.
High-level energy management
• Sophisticated capabilities such as proactivity and self-tuning
increase the energy demand of software on a mobile
computer in one’s personal computing space.
• Making such computers lighter and more compact places
severe restrictions on battery capacity, so the higher levels
of the system must be involved in memory management.
• One example is energy-aware memory management, where
the operating system dynamically controls the amount of
physical memory that has to be refreshed.
• Another example is energy-aware adaptation, where
individual applications switch to modes of operation with
lower fidelity and energy demand under operating system
control.
Client thickness (or client’s hardware
capabilities)
• For a given application, the minimum acceptable thickness of
a client is determined by the worst-case environmental
conditions under which the application must run
satisfactorily.
• A very thin client suffices if one can always count on highbandwidth low-latency wireless communication to nearby
computing infrastructure, and batteries can be recharged or
replaced easily.
• If there exists even a single location visited by a user where
these assumptions do not hold, the client will have to be
thick enough to compensate at that location.
• This is especially true for interactive applications where
crisp response is important.
Context awareness
• A pervasive computing system must be cognizant of its
user’s state and surroundings, and must modify its
behavior based on this information.
• A user’s context can be quite rich, consisting of various
attributes
– Ex) physical location, physiological state (e.g., body temperature and
heart rate), emotional state (e.g., angry, distraught, or calm), personal
history, daily behavioral patterns, and so on.
• If a human assistant were given such context, he or she
would make decisions in a proactive fashion, anticipating
user needs.
• In making these decisions, the assistant would typically
not disturb the user at inopportune moments except in
an emergency.
• A pervasive computing system should emulate such a
human assistant.
Balancing proactivity and transparency
• Unless carefully designed, a proactive system can annoy a
user and thus defeat the goal of invisibility.
• A mobile user’s need and tolerance for proactivity are likely
to be closely related to his/her level of expertise on a task
and familiarity with his/her environment.
• A system that can infer these factors by observing user
behavior and context is better positioned to strike the right
balance.
• For transparency, a user patience model can be implemented
to predict whether the user will respond positively to a
fetch request.
– So the user interaction is suppressed and the fetch is handled
transparently.
Impact on layering
• Proactivity and adaptation based on corrective actions
seem to imply exposure of much more information
across layers than is typical in systems today.
• Layering cleanly separates abstraction from
implementation and is thus consistent with sound
software engineering.
• Layering is also conducive to standardization since it
encourages the creation of modular software
components.
Privacy and trust
• As a user becomes more dependent on a pervasive computing
system, it becomes more knowledgeable about that user’s
movements, behavior patterns and habits.
• Exploiting this information is critical to successful proactivity and
self-tuning (invisibility), but also may cause serious loss of privacy.
• User must trust the infrastructure to a considerable extent
and the infrastructure needs to be confident of the user’s
identity and authorization level before responding to his/her
requests.
• It is a difficult challenge to establish this mutual trust in a
manner that is minimally intrusive and thus preserves
invisibility.
Charting Past, Present, and
Future Research in Ubiquitous
Computing
Gregory Abowd and Elizabeth Mynatt
ACM TOCHI 2000
•
•
•
•
•
Natural interfaces
Context-aware computing
Automated capture and access
Everyday computing
System evaluation challenges
Computing with natural interfaces
• Ubicomp inspires “off-the-desktop” applications
– Needs “off-the-desktop” means of interaction
– Speech, gestures, writing
• More accessible
• Easier to use???
Computing with natural interfaces
• Need for first-class natural data types
– To ease the development of more apps w/
natural interfaces
• Should be easy keyboard and mouse
– Support for “fundamental” operations
• E.g., text and speech recognitions
– Support for “extra” operations
• Tivoli Live Board: special input like structured
gestures such as dragging and freeform selection
• Classroom 2000: annotating pen strokes with
timestamps
Computing with natural interfaces
• Error prone interaction
– Permit new and numerous mistakes
– People do not have perfect recognition
• As low as 54%; cursive handwriting 88%; printed handwriting
96.8%
– Recognition accuracy == user satisfaction??
• Not really: complexity of error recovery dialogues and valueadded benefit of any given efforts
• Entering a command vs. writing journal entries
• Several research areas
– Error reduction (about 5-10%)
– Error detection
– Reusable toolkit for error handling
Context aware computing
• Current Systems
– Generally using position and identification of
objects
– Still do not provide a complete context
– Definition of context is limited
• Research areas
– Context toolkits
• Toolkit for sensing environment
• Explicit use of sensed information is up to program
– What is context?
– How is context represented?
What is context?
•
Who
– Currently generally tailored to one user
– How important are others in determining our behavior
• How could this be captured?
•
What
– Attempt to figure out what is currently happening
– Sense environment, use calendar software etc.
•
Where
– Location based information, e.g., GPS
– Most explored context information
•
When
– Easily obtained information -- Computer is good at remembering time
•
Why
• Although determining when one event stops and another begins is not easy
– Even harder than the “what” question, biometric sensors might help (e.g.,
body temperature, heart rate, etc)
Toward context aware computing
• Context representation
– Requires universal context schemes or toolkits with
standard context representations
• Context sensing and fusion
– How to make context-aware computing “ubiquitous”?
– In practice, there are few truly ubiquitous, singlesource context services
• E.g., GPS does not work indoors; different indoor localization
schemes have different characteristics (e.g., cost, range)
– Like sensor fusion, context fusion handles seamless
handling of sensing responsibility between boundaries
of different context services
– Combining multiple context sources can increase the
accuracy of context information
Automated capture and access
• Recording information and data as it occurs
– Computers are inherently good at recording, people are not
– People freed up to summarize and understand
– Most work in academic/ classroom settings
• Time stamping lectures, digital whiteboards
• Challenges in “capture and access”
– Sometime we don’t know we want to capture something until after
its already happened
• How could the computer know that?
• If it captures everything then we need a system of sorting and
filtering (access)
– Gmail Problem
– Access is a problem because capturing of raw data can be
burdensome for sifting through; systems need to recognize
important events facilitate access
Everyday computing
• Continuous interactions (i.e., no clear beginning or end)
– Both fundamental activities like communication and long-term
endeavors do not have predefined starts and ends; information
from past can be recycled
– Very different traditional HCI design which assumes “closure” with
clear goals like spell checking, dialogue, etc.
• Interruption is expected:
– People are constantly interrupted
– Computer systems must recognize interruption and change state
• Also computers must appropriately inform users
• Multiple activities operate concurrently:
– People multitask and rapidly switch task based on external
unpredictable environment
– Systems need to adapt to this opportunistic behavior and change
accordingly
Toward everyday computing
• Develop continuously present interface
– No current model of continuously present interfaces, even
people are not continuously present
– Create an interface that doesn’t get annoying (e.g., wearable
devices)
• Determine what information should require my
attention and what should be display peripherally
• Connect events in the physical and virtual worlds (e.g.,
face to face vs. email, document, webs)
• Modify/fuse existing HCI schemes to efficiently
support everyday computing (but evaluation is
challenging and laborious)
System evaluation challenges
• Hard to evaluate Ubicomp Systems
– Little publish on ubicomp evaluation
– Systems often required to be fully connected leading to
systems that are hard to build
– Lack of development toolkits make system creation difficult
– Systems often need to be integrated into peoples lives which
using big clunky prototypes does not lead itself well too
– Task/Goal centric approaches don’t work in ubicomp
Example Projects
• Pervasive computing projects have emerged at major
universities and in industry:
–
–
–
–
–
Project Aura (Carnegie Mellon University)
Oxygen (Massachusetts Institute of Technology)
Portalano (University of Washington)
Endeavour (University of California at Berkeley)
Place Lab (Intel Research Laboratory at Seattle)
Example Projects : Project Aura (1)
• Aura (Carnegie Mellon University)
– Distraction-free (Invisible) Ubiquitous Computing.
Example Projects : Project Aura (2)
• Moore’s Law Reigns Supreme
–
–
–
–
–
–
Processor density
Processor speed
Memory capacity
Disk capacity
Memory cost
...
Human Attention
• Glaring Exception
– Human Attention
Adam & Eve
2000 AD
Example Projects : Project Aura (3)
• Aura Thesis:
– The most precious resource in computing is human attention.
• Aura Goals:
– Reduce user distraction.
– Trade-off plentiful resources of Moore’s law for human
attention.
– Achieve this scalably for mobile users in a failure-prone,
variable-resource environment.
Example Projects : Project Aura (4)
• The Airport Scenario
– Jane wants to send e-mail from the
airport before her flight leaves.
• She has several large enclosures
• She is using a wireless interface
– She has many options.
• Simply send the e-mail
– Is there enough bandwidth?
• Compress the data first
– Will that help enough?
• Pay extra to get reserved bandwidth
– Are reservations available?
• Send the “diff” relative to older file
– Are the old versions around?
• Walk to a gate with more bandwidth
– Where is there enough bandwidth?
– How do we choose automatically?
Example Projects : Project Aura (5)
• The Mobile Task Scenario
– Aura saves Scott’s task.
– Scott enters office and gets strong
authentication and secure access.
– Aura restores Scott’s task on
desktop machine and uses a large
display.
– Scott controls application by voice.
– Bradley enters room.
– Bradley gets weak authentication,
Scott’s access changes to insecure.
– Aura denies voice access to sensitive
email application.
– Scott has multi-modal control of
PowerPoint application.
– Aura logs Scott out when he leaves
the room.
Example Projects : Oxygen
• Oxygen (MIT)
– Pervasive human-centered computing.
– Goal of Oxygen is bringing abundant computation and
communication, as pervasive and free as air, naturally into
people's lives.
Example Projects : Oxygen (2)
• To support highly dynamic and varied human activities, the
Oxygen system must be
– pervasive— it must be everywhere, with every portal reaching into the same
information base;
– embedded— it must live in our world, sensing and affecting it;
– nomadic— it must allow users and computations to move around freely,
according to their needs;
– adaptable— it must provide flexibility and spontaneity, in response to
changes in user requirements and operating conditions;
– powerful, yet efficient— it must free itself from constraints imposed by
bounded hardware resources, addressing instead system constraints imposed
by user demands and available power or communication bandwidth;
– intentional— it must enable people to name services and software objects by
intent, for example, "the nearest printer," as opposed to by address;
– eternal— it must never shut down or reboot; components may come and go in
response to demand, errors, and upgrades, but Oxygen as a whole must be
available all the time.
Related Projects: Portalano
• Portolano (University of Washington)
– An expedition into invisible computing.
– Expedition goals:
• Connecting the physical world to the world-wide information fabric
– Instrument the environment: sensors, locators, actuators
– Universal plug-and-play at all levels: devices to services
– Optimize for power: computation partitioning, comm. opt.
– Intermittent communication: new networking strategies
• Get computers out of the way
– Don’t interfere with user’s tasks
– Diverse task-specific devices with optimized form-factors
– Wide range of input/output modalities
• Robust, trustworthy services
– High-productivity software development
– Self-organizing, active middleware, maintenance, monitoring
– Higher-level, meaningful services
Related Projects: Portalano (2)
• Scenario
– Alice begins the day with a cup of coffee and her
personalized newspaper.
– When her carpool arrives, she switches to reading the news
on her handheld display, where she notices an advertisement
for a new 3-D digital camera.
– It looks like something that would interest her shutterbugfriend Bob, so Alice asks her address book to place the call.
Related Projects: Portalano (3)
• Scenario (2)
– Bob's home entertainment system softens the volume of his
custom music file as his phone rings.
– Alice begins telling Bob about the camera, and forwards him
a copy of the advertisement which pops up on his home
display.
– Bob is sold on the product, and after hanging up with her, he
asks his electronic shopping agent to check his favorite
photography stores for the lowest price and make the
purchase.
Related Projects: Portalano (4)
• Scenario (3)
– When the camera arrives, Bob snaps some photos of his
neighbor's collection of antique Portuguese navigation
instruments.
– After reviewing the photo album generated automatically by
a web-based service, Bob directs a copy of his favorite image
to the art display in his foyer.
– He also sends a pointer to the photo album to Alice and
instructs his scheduling agent to set up a lunch date so that
he can thank her for the suggestion.
Example Projects : Endeavour
• The Endeavour Expedition (UC Berkeley)
– Charting the Fluid Information Utility
• Endeavour Goal:
– Enhancing human understanding through the use of
information technology.
•
•
•
•
•
Principles of Pervasive Computing
Evolution & Related Fields
Example Projects
Other Scenarios
References
Other Scenarios
• Buy drinks by Friday (1)
– Take out the last can of soda
– Swipe the can’s UPC label, which
adds soda to your shopping list
– Make a note that you need soda
for the guests you are having
over this weekend
Other Scenarios
• Buy drinks by Friday (2)
– Approach a local supermarket
– AutoPC informs you that you are
near a supermarket
– Opportunistic reminder: “If it is
convenient, stop by to buy
drinks.”
Other Scenarios
• Buy drinks by Friday (3)
- Friday rolls around and you have
not bought drinks
- Deadline-based reminder sent to
your pager
Other Scenarios
• Screen Fridge
– Provides:
•
•
•
•
•
•
•
•
•
Email
Video messages
Web surfing
Food management
TV
Radio
Virtual keyboard
Digital cook book
Surveillance camera
Other Scenarios
• The Active Badge
– This harbinger of inch-scale computers contains a small
microprocessor and an infrared transmitter.
– The badge broadcasts the identity of its wearer and so can
trigger automatic doors, automatic telephone forwarding and
computer displays customized to each person reading them.
– The active badge and other networked tiny computers are
called tabs.
Other Scenarios
• The Active Badge
Other Scenarios
• Edible computers:
The pill-cam
– Miniature camera
– Diagnostic device
– It is swallowed
• Try this with an
ENIAC computer!
Other Scenarios
• Artificial Retina
– Direct interface
with nervous
system
– Whole new
computational
paradigm (who’s the
computer?)
Other Scenarios
• Smart Dust
– Nano computers that couple:
• Sensors
• Computing
• Communication
– Grids of motes (“nano
computers”)