Keynote Presentation
Pervasive Computing:
Towards Self-Aware Systems
Volker Christian
Johannes Kepler Universität Linz, Austria
voc@soft.uni-linz.ac.at
1. INTRODUCTION
The last decade of the past century has changed the world more than any decade before.
This change was mostly enforced by technological advances in information and
communication technologies.
Observably, information and communication technology has found its place in our
everyday life. Most of us are equipped with at least a cellphone and more and more often
by some kind of Personal Digital Assistant (PDA).
In the same way as the potential and the power of those devices has grown up and their
size has been reduced, they have become cheaper and cheaper. The consequence of that
fact is that the number of those devices has enormously grown since 1990. They could be
found in many different objects of everyday life. And in the near future, it will be
completely normal for us to be surrounded by a huge number of new evolving smart
gadgets, assisting us in day-to-day processes [Bohn et al. 2003].
The level of apparentness of such support will differ for different applications and
different types of devices. Some will be completely hidden from the user and some others
would be located more prominently in the environment.
However, the kind of interaction with those visionary devices and systems would be
unfamiliar for us from the present point of view. In contrast to the way of traditional
interaction based on keyboard, mouse, and display, we will use human native interaction
mechanisms like gestures, voice, or even more elaborated, facial expressions [Wellner et
al. 1993]. Thus, no direct interaction with technology would happen – the devices observe
and collect information about humans during natural situations and affectations and
intelligent software is responsible for drawing conclusions out of it. These conclusions lead
to reactions which, via various different actuators, are affecting the real physical world
again. This “information-loop” - sensing, analyzing, and reacting - would generate highly
complex scenarios completely new in computer science.
On the other hand, rendered possible due to advances in network and sensor technology,
a complete new feature, namely awareness, is just evolving. Devices independently sense
and collect information about their environment using various different technologies like
Global Positioning System (GPS), Radio Frequency Identification tags (RFID) systems,
magnetic and optical tracking systems, biometric and biological sensors, but contribute
with that information to the knowledge of a collaborative system. This collaborative
system is formed by those devices itself interconnected by modern network technologies
like e.g. Global System for Mobile Communication (GSM), Universal Mobile
Telecommunication System (UMTS), Wireless LAN (WLAN), Bluetooth, Infrared, and
other radio based technologies. Such huge connected systems of many individual devices
additionally equipped with an adequate software system would form a meta-system
showing a self-aware behavior in analogy with biological meta-systems like ant-hills or
fish swarms for example.
The study of the structure, the construction, and the modeling of the behavior and the
characteristics of such interconnected sensor-equipped systems is the domain of Pervasive
Computing, expecting to introduce a new quality into human computer interaction and a
grant of unobtrusive support of humans during their everyday life.
2. MOORE'S LAW AND ITS IMPACT ON PERVASIVE COMPUTING
Pervasive Computing would not be possible without the availability of very small but yet
powerful systems. This has been fertilized by the process of constant miniaturization of
hardware along with an increase in performance.
1965, Gordon Moore has observed that the number of transistors to be able to be
integrated error-free on a square inch of silicon-tie almost doubles every 12 month [Moore
1965]. This legality known as “Moore's law”, flatters somewhat to 18 month per
duplication in the years after 1965. However, since 1970 this rate is more or less constant.
Along with the reduction of the structure size on silicon, also the speed of the chips is
continuously growing. Due to already perceivable further technologies like chips based on
quantum mechanic principles, it is supposed that Moore's law would be valid for the next
20 years. Based on that period of time and the doubling rate one could calculate, that the
performance of devices would increase by a factor of 16384. This enormous increase will
make it possible to build much smaller but much more powerful devices in the future.
Beside the almost established integration of computer hardware into consumer electronics,
vehicles, home appliances, etc. we will find some kind of computer hardware in almost
every object in the future – clothes, foot, walls, doors, and possibly implantations even in
the human body itself [Schmidt & Laerhoven 2001].
3. THE SOFTWARE CHALLENGE
As clear as the future could be seen for hardware at least for the next view years, as diffuse
it is for the software. It is a known fact that software didn't evolve as fast as hardware. But
especially in this new situation of an highly visible number of devices embedded into
physical objects forming a smart environment, the situation for software is an even bigger
challenge. The well established client/server network architecture is presumably not the
ideal solution in this new situation. Scalability and maintainability appear to be one of the
main problems which immediately emerge.
Many different projects have evolved to investigate and to cope with that new situation
and different nomenclatures describing more or less the same quality of approaches could
be found in literature. Pervasive Computing, Ubiquitous Computing, Ambient Computing,
Calm Computing, Sentient Computing, Autonomous Computing, Hidden Computing,
Invisible Computing, and Disappearing Computing – all of them have the same guiding
idea: “The most profound technologies are those that disappear. They weave themselves
into the fabric of everyday life until they are indistinguishable from it”, formulated by
Mark Weiser from the Xerox PARC group [Weiser 1991]. And all of them break with the
old client/server based computing and propose new architectures based on spontaneous adhoc peer to peer technologies. Even the big number of projects and project titles reflects
the big effort put into those new software systems.
Today, the term “Pervasive Computing” has become most widely accepted describing
in short the guideline of the just evolving paradigm: Reduced to functionality, device
independent, intelligent information technology which is not identifiable as technology but
providing background assistance which acts proactively and largely autonomously.
4. COMPUTATIONAL PERCEPTION AND CONTEXT
The interaction with smart environments where everything is connected to everything is of
fundamental importance. It is obvious that the established interaction technologies like
keyboard, mouse, and display are not appropriate for pervasive computing systems. These
interfaces are replaced by a potpourri of diverse sensors and modern technologies like
RFID tags, GPS, temperature sensors, cameras, microphones, acceleration and rotation
sensors, to note only some of them.
It is a fundamental must for such systems, which are based on those new interfaces
some kind of computational perceptional is integrated into them. Autonomous cognition,
localization, perception, and prediction of future situations would be essential and
necessary abilities of software components for pervasive systems.
The goal is to develop a pervasive computing environment which could collect
information about its associated physical environment by use of all those different sensors
and which could generate a representation of the direct and indirect context out of that
sensory informations, interpret this context representation and act back into the real world
controlled by an automatically generated context prediction [Ferscha 2003].
“Context” as a term in software engineering has been introduced and defined by
different groups. A.K. Dey, for example, defines context as “any kind of information
necessary to characterize the situation of an entity. An entity is a person, place, or object
that is considered relevant to the interaction between a user and an application, including
the user and applications themselves” [Dey 2000]. It is not specified in more detail what an
entity is – in fact, an entity is a placeholder for an arbitrary physical object, e.g. Mahadma
Ghandi, a rubber, a rose, etc. Other projects have done a categorization on top of the
entities. In the Cooltown Project the categories “People”, “Place”, and “Thing” are
introduced and postulated to be sufficient for most applications [Kindberg et al. 2000].
The term “context-aware” in general describes the ability of a system to recognize and
appoint objects and persons along with their intentions.
Furthermore, the Xerox Parc Group defines the term “context-aware computing” as “the
ability of a mobile user's application to discover and react to changes in the context in
which they are situated” [Schilit 1995].
To advance towards the aim of developing context-sensing and interpreting
applications, the process of perception has to be formalized and structured in a general way
to be easily and flexible applicable during software development. This formalization is the
domain of the field of “Computational Perception”. Combined with the field of “Pervasive
Computing”, one gets context-aware and context-based applications able to integrate
themselves into and cooperate with the natural human environment.
As far as the context-information collecting sensors permit the digital representation of
context at a particular moment is at the same time a digital snapshot of the physical world
automatically. This snapshot combined with context-information interpretation and context
prediction could be viewed as the virtual representation of a psychological cognitive
process.
5. EVERYWHERE AND NATURAL INTERFACES
The consequent incorporation of all human senses, not just audio and visual perception,
and the rigorous replacement of traditional interfaces like keyboard, mouse, and display, is
the central challenge of the field of “Everywhere and Natural Interfaces”. Even adding
speech-processing and speech-recognition to an application doesn't cover all the possible
human computer interaction. Gesture, emotion, facial expression, habits, grasping, etc. are
all potential interfaces to embedded systems and enable intuitive alternative interfaces to
smart, cooperative applications. The challenge is to make those new interfaces situational,
intelligent, and autonomous to make the ambient digital environment easily and intuitively
accessible to humans, everytime and everywhere.
6. Context Aware Frameworks and Applications
By formalizing the whole process of
context-processing, six different layers
could be identified and specified [Beer
2003].
1.
2.
3.
4.
5.
6.
Context Sensing
Context Aggregation
Context Representation
Context Interpretation
Context Prediction
Context Controlled Acting
A variety of software frameworks
have been developed supporting this
layer structure to help developing
context-aware
applications
[Schilit
1995][Kindberg et al. 2000][Dey 2001].
The first layer, Context Sensing, is
responsible for collecting as much
information as possible about the context
Sensor
Sensor
Sensor
Sensor
Sensor
Sensor
Context Aggregation
Context Representation
Context Interpretation
Context Prediction
Actuator Actuator Actuator Actuator Actuator Actuator
Typical Architecture of a Context-Aware Application
of the physical environment, to transform this information into a consistent collective data
format, and to forward this data to the next layer. During Context Aggregation, all data
coming from the Context Sensing layer are collected and combined in a way, that the
Context Representation could generate a unified abstraction of the real world. The Context
Interpreter together with the Context Prediction takes this abstraction and performs some
application specific transformations to provide control values to perform Context
Controlled Acting.
The challenge in developing context-applications following this architecture is the
modeling of the interpretation of the context representation in terms of the semantics of the
application. While earlier approaches have modeled context by some sort of key values,
new approaches are using meta-data, object orientation models, or logic orientated systems
where context are facts in rule based compositions.
No satisfying general solution has been found so far for the provision of proactivity for
context-aware applications. Applications following a state-based context model approach
need some sort of prediction component to adjust the application, to meet some potential
future context configurations. Here the triggered action is not just controlled by a previous
context representation but it also takes into account the not yet achieved but conceivable
future state.
Applications equipped with interpretation and prediction systems would have the ability
to anticipate the future and to accomplish actions aimed at an anticipated future context
configuration. Proactivity would enable context-aware systems to rate the consequences of
its activity in the physical world. By treating this rating as the input for a tuning process of
the application and the prediction module itself, one would end with a process comparable
to learning of creatures, for which some level of self-awareness is a necessary
precondition.
7. The Future
Beside the open questions already mentioned above, there are many more not yet solved
challenges not discussed so far. Some of them are of technical nature while others are of
more political and juridical character.
●
Everywhere Access: Guaranteeing time, location, and device independent access to
relevant information on the bases of wireless personal, local, and global communication
technologies.
●
Security: Due to the enormous increase of the number of independent but connected
devices and the heterogeneity of software and operating systems, security will be of
even higher priority than today. A virus-attack would have a dramatical impact on a
connected system of small devices where, due to the enormous number of such devices
and the lack of direct interfaces like keyboards and displays, nobody is able to react fast
enough to take some counteractive measures. The whole system has to be equipped with
some sort of immune system able to react autonomously to adversary attacks.
●
Privacy: Some new techniques have to be developed to preserve the data-highness of
individuals. Iris-scanner, fingerprint-scanner, and even more sophisticated technologies
are promising to guarantee absolute safety that personal data could only be accessed by
authorized persons. Also basic juridical and political conditions have to be developed to
guarantee order and safeness in this evolving new environment.
We have already gone a long way towards the realization of the idea of embedded
interconnected devices forming a smart self-aware environment. But many problems are
left open. Further work has to be done until we really live in a world where such systems
would be reliable and treated as normal as the traditional television set today, assisting us
in everyday processes. With the support of Moore's law, we could expect to converge to
this vision soon.
8. Literature
[Bohn et al. 2003] Bohn, J.; Coroama, V.; Langheinrich, M.; Mattern, F.; Rohs, M.: Allgegenwart und Verschwinden
des Computers – Leben in einer Welt smarter Alltagsdinge. In: Ralf Grötker (Hrsg.): Privat! Kontrollierte Freiheit in
einer vernetzten Welt. Heise-Verlag, S. 195-245, 2003.
[Beer 2003] Beer, W; Christian, V; Ferscha, A; Mehrmann, L.: Modeling Context-Aware Behavior by Interpreted ECA
Rules, Euro-Par 2003, Springer Verlag, LNCS 2790, pp. 1064-1073, 2003.
[Dey 2000] Dey, A.K.: Providing Architectural Support for Building Context-Aware Applications. PhD thesis, Georgia
Insititut of Technology, 2000.
[Dey 2001] Dey, A.K.; Salber, D.; Abowd, G.D.: A Conceptual Framework and a Toolkit for Supporting the Rapid
Prototyping of Context-Aware Applications, Human-Computer Interaction (HCI) Journal, Volume 16 (2-4), 2001, pp.
97-166.
[Ferscha 2003] Ferscha, A.; Vogl, S.; Beer, W.: Context Sensing, Aggregation, Representation and Exploitation in
Wireless Networks, Future Generation Computing Systems, North Holland, 2003
[Kindberg et al. 2000] Kindberg, T.; Barton, J.; Morgan, J.; Becker, G.; Bedner, I.; Caswell, D.; Debaty, P.; Gopal, G.;
Frid. M.; Krishnan, V.; Morris, H.; Schettino, J.; Serra, B.; Spasojevic, M.: People, Places, Things: Web Presence for
the Real World. WWW'2000, 2000.
[Moore 1965] Moore, G. E.: Cramming More Components Onto Integrated Circuits. Electronics, 1965.
[Schilit 1995] Shilit, W.N.: A system architecture for context-aware mobile computing, Order Number: UMI Order No.
GAX95-33659, Columbia University, New York, 1995.
[Schmidt & Laerhoven 2001] Schmidt, A.; Laerhoven, K.: How to Build Smart Appliances?, IEEE Personal
Communications, Vol. 8, No. 4, pp. 66-71, 2001.
[Weiser 1991] Weiser, M.: The Computer of the Twenty-First Century. Scientific American, pp. 94-100, 1991.
[Wellner et al. 1993] Wellner, P.; Mackay, W.; Gold, R.: Computer Augmented Environments: Back to the Real
World. CACM, Vol. 36, No. 7, 1993.