Context Acquisition Chen & Kotz: context awareness1\survey of CAMobileC.pdf Albrecht_Schmidt:G:\context awareness1\Albrecht_Schmidt_PhDThesis_Ubiquitous-Computing_ebook1.pdf Schmidt:G:\context awareness1\Schmidt-how to build smart appliances.pdf Chen:G:\context awareness1\intelligent broker.pdf Gellersen:context awareness1\monet.pdf Sensing the Context (1/3) Location: Outdoors: GPS Indoors: IR, RF, ultrasonic, camera (cellular and non-cellular) Hybrid: IEEE 802.11, Mobile-IP Sensing the Context (1/3) Issues: Heterogeneous sensors with uncertainty and conflicts (sensor fusion) Data networks vs sensor networks communication and positioning may be combined or independent. (Active Badge & ParcTab use the same wireless infrared link for both purpose; and the RF link of the GUIDE and RADAR systems has the same role) it is necessary to decouple the positioning and communication channels. Sensing the Context (1/3) Making mobile devices location-aware the system tracks the location by monitoring beacons from mobile devices and the mobile device queries a central database to get the current location the mobile device passively listens to beacons from the cell basestation and queries a local database for its current location. In the latter case, if the mobile device only queries a local database for location, it has complete privacy and it can choose to advertise its current location to the world or only to selected third parties. Sensing the Context (2/3) Low-level contexts beyond location Time: time-of-day (with calendar) Nearby objects Network bandwidth Orientation Others: photodiode (light), accelerometer (tilt, vibration), microphone, sensors for temperature, pressure, gas, etc. Issue: sensors in mobile devices or infrastructure => direct vs. indirect awareness Sensing the Context (3/3) High-level contexts: user’s activity Camera technology and image processing Consult calendar for what user is to do Combine low-level sensors, e.g., using rules How about emotional contexts? Context changes: subscriptionnotification Polling rate? Quality of Contexts What context is important? Always and in different situations? Quality: Coverage, resolution, accuracy, confidence, reliability, frequency, timeliness Self-contained vs. infrastructure-supported PDA doesn't need location sensors if it can ask nearby sensors to approximate Need standards for sharing components? Context vs. Situation “In a certain situation, specific sensor data is frequently similar to sensor data of the same situation at different times.” Situation: the state of the real world at a certain moment or during an interval in time at a certain location. Context: identified by a name and includes a description of a type of situation by its characteristic features. Situation S belongs to a Context C: A situation S belongs to a context C when all conditions in the description of C evaluate to true in a given situation S. Requirements on Sensing in a Ubiquitous Computing Environment Design and Usability Energy Consumption Calibration Start-up Time Robustness and Reliability Portability, Size and Weight Unobtrusiveness, Social Acceptance and User Concern Price and Introduced Cost Precision and Openness Sensing Technologies and Systems for Data Capture Light and Vision Audio Movement and Acceleration Location and Position Magnetic Field and Orientation Proximity, Touch and User Interaction Temperature, Humidity and Air Pressure Weight Motion Detection Gas-Sensors and Electronic Noses Bio-Sensors Zero-Power Sensors Composition of Sensing Systems All sensors at same position Homogeneous e.g. orientation sensing aware PDA, system [Schmidt,98] Heterogeneou s sensing system Sensors distributed e.g. load sensing system, [Schmidt,02] e.g. context-aware e.g. distributed mobile phone, sensing boards, [Schmidt,99c] [SMART,02] Homogeneous sensing system Microphone array light sensors in a handheld device to avoid user’s finger put on top of the sensor resulting in a false reading. multiple sensors of the same type placed at different physical positions. Especially when there are little restrictions in the way a device is held or carried Placement of sensors Placement of sensors is of major importance and has a great influence on the quality of the data that is gathered. putting all sensors together in one place, perception of contexts becomes much harder than for a case where sensors are physically distributed and placed at an optimal point; especially true for applications in wearable computing. Finding and selecting the ‘right’ position for a sensor is very much dependent on the contexts that should be recognised. In most case, there is no prior knowledge about sensor placement try and error simulation/emulation tools Perception Methods Basic Statistical Functions: average, median, standard deviation, max-min range, interquartile range, the amount of change Time Domain Analysis: data from accelerometers, light sensors and audio Derivatives: of interest to find information about the change rather than about the absolute values Neural Networks: to provide abstract or symbolic information (cues) Rule Based Systems:particularly simple when sensors map well to contexts of interest and the number of contexts is small A Perception Architecture Albrecht_Schmidt:G:\context awareness1\Albrecht_Schmidt_PhDThesis_Ubiquitous-Computing_ebook1.pdf Sensor layer Cue layer Context layer Context Acquisition Design Method Step 1. Identifying contexts that matter Is the application or artefact used in changing situations? Vary the expectations of the user towards the application or artefact with the situation? Is the human computer interaction pattern different in various situations? Does the user’s interaction with the real world environment offer information that is valuable to the application? Context Acquisition Design Method Step 2. Determine variables that distinguish contexts selected. Such variables may be of informational, physical and social nature (e.g. location, temperature, relationship with people near by, interaction with the device) For the minimal set of variables that make it possible to discriminate the selected context the values (e.g. ranges, sets) for variables are specified. Context Acquisition Design Method Step 3. Finding appropriate sensors to cover all variables at minimum cost. accuracy of the sensor in relation to the variable the cost to provide the information feasibility and appropriateness The resulting selection of sensors should be done in a way that the sensors cover all variables with sufficient accuracy at a minimal cost. Context Acquisition Design Method Step 3a. Building and assessing a prototypical sensing device. (optional) when there is little knowledge about the sensors in question or the contexts to recognise. can experiment with the positions of the sensors on the device. the recorded data is analyzed to identify whether the raw data differs significantly for the different situations or not. If the data differs not significantly different sensors have to be selected or even different contexts have to be identified or in some case it may turn out that it is not feasible to recognise the contexts at all. Context Acquisition Design Method Step 4. Selecting recognition and abstraction technologies. Step 5. Specification of a context acquisition system After all components, the physical design and the algorithms are assessed the overall architecture of the system is determined. Centralized vs. distributed Build applications. Context Acquisition Design Method Context Acquisition Library: G:\context awareness1\Albrecht_Schmidt_PhDThesis_Ubiquitous-Computing_ebook1.pdf Rapid Prototyping Platform for Context Acquisition: Smart-Its Cost function Power Consumption、Size and Weight、Price of Components、Robustness and Reliability Context Acquisition method— direct access to hardware sensors Chen:G:\context awareness1\intelligent broker.pdf A key benefit of this approach is that the high-level applications can have great controls over the operations of the lowlevel sensors, and can have better knowledge about how different data is collected and computed. Context Acquisition method— direct access to hardware sensors Shortcomings: To communicate with different sensors means an application must maintain the implementation to communicate with different sensors. As the number of context used by the application increases, the amount of implementation for sensor communications also increases. For this reason, the application’s overall implementation becomes difficult to maintain. Facilitated by Middleware Using middleware, context-aware applications’ implementations can focus on how to use context but not on how to acquire context. middleware is typically built into the hosting devices or platform on which the context-aware applications operate. Odyssey project: context aware agents are built on a middleware to acquire status about the communication network; demonstrating three different context-aware agents that can adapt their behaviors according to the network bandwidth changes. Context Toolkit: aimed to provide a general solution for building scalable and reusable context acquisition modules; built on the widget concept in the graphical user interface design. Facilitated by Middleware Problem: imposes additional computation burden on the hosting devices. A middleware design trades computation resources for development convenience. In order to maintain a generic programming interface between the high-level applications and low-level sensors, certain amount of computation resources (e.g., CPU power, memory, network bandwidths) must be allocated for the middleware operations. While this might not create a problem for devices that have rich computing resources, however, it will lead to resource contention problem in devices that have less resources (e.g., cellphones and embedded devices). Acquiring Context from a Context Server For the case that the hosting device has limited computing resource context server: a computing entity that provides contextual information to different context-aware applications in a distributed environment. to shift the context acquisition procedures into the implementation of a server entity that runs on a resource-rich device. applications that do not have built-in sensing capability can acquire context and become context-aware. Acquiring Context from a Context Server Me-Centric Domain Server [Perich]: the physical world is divided into a set of microworlds. Each micro-world represents a particular domain in the physical world. A domain might be an office room, a meeting room, etc. Each microworld has context. The role of a Domain Server is to maintain the context of the individual microworlds and share this information with the computing entities of a Me-Centric application. CoBrA Similarity: a server entity is responsible to share contextual information with other agents in the space; use knowledge representation languages to express contextual information (i.e., CoBrA uses OWL, and Me-Centric Domain Server uses RDF). Acquiring Context from a Context Server Differences: the CoBrA is not only to share contextual information but also to maintain a consistent model of the context to detect and resolve inconsistent information that may have been acquired from unreliable sensors and agents. Using OWL to represent context as ontologies, CoBrA allows independently developed agents to share knowledge and provides a means for context reasoning. CoBrA addresses the privacy issue by providing users with a policy language for controlling the sharing of their contextual information. Privacy was not addressed in the Me-Centric Domain Server. Multi-Sensor Context-Awareness in Mobile Devices and Smart Artefacts Gellersen:context awareness1\monet.pdf Position Sensing and Location-based Context Auditory and Visual Context Other Sensors: bio-sensors, tilt sensors Integration of Diverse Sensors Multi-Sensor Context-Awareness in Mobile Devices and Smart Artefacts TEA – Technology Enabling Awareness: investigated the realization of multi-sensor context-awareness in a self-contained device that would be available as peripheral or plug-in for mobile host devices. The general application perspective was to supply situational context to a mobile host device to improve the device’s service to its user. We explored this perspective in the application domain of mobile telephony. Multi-Sensor Context-Awareness in Mobile Devices and Smart Artefacts Mediacup: looked at how non-computational artifacts can be augmented with awareness technology. The application perspective is entirely different from that underlying TEA. Awareness is not employed to improve the immediate function of the augmented device, but to create a digital presence for it. New functionality is not expected to emerge in the device itself but in the surrounding system environment. Smart-Its: toward platforms for aware mobile devices. With this project we also shift our attention from individual devices to ad hoc networking of aware devices, and scenarios of collective awareness. Manipulating Multiple Contexts Source: “Multi-Sensor Context-Awareness in Mobile Devices and Smart Artefacts” H.W. Gellersen, A. Schmidt, M. Beigl Lancaster University and University of Karlsruhe Situation Context perceived sensor(s) Direct/Indirect Context-Awareness Indirect context awareness: device A Situation Context Infrastructure perceived sensor(s) device B communicate device C Direct context awareness: Situation Context perceived S(s) device A S(s) device B S(s) device C Built-in sensors, context processing Single Sensor vs. Multiple Sensors Single but powerful sensor: Position sensor, video sensor (camera) Useful information is inferred from what is perceived Prior knowledge is needed for information useful • Simple but cheap sensors: •Each sensor captures one facet of the context Take Multiple-Sensor Approach Need to combine multiple simple sensors vs. one camera with powerful recognition capability Gain rich data to infer useful context with little computation Real world situations: situations and sensors Example: TEA Technology Enabling Awareness Motivation: make personal mobile devices smarter host Specs: CPU: PIC16F877 Storage: 8K EEPROM RAM: 200 Byte Use serial line to communicate with the host TEA TEA Architecture Cue: abstraction of raw data For example, an acceleration sensor can infer cues like pattern of movement and current speed Rules from cue to context: can be pre-defined or use supervised/unsupervised learning Initial Exploration Tried on both PDA and mobile phone, and found it more useful on mobile telephony Two analysis strategies: Analyze how well a sensor contributes to a given context Analyze which sensors relate to which context Finding: audio, motion and light are useful globally, and other sensors are only useful in specific contexts Implementation Some data are directly analyzed and not stored (audio) Some data are stored and processed (acceleration, light) Some data are cues itself; no need to process (temperature, skin conductance) Rules from cue to context are extracted offline and hardwired Application Profile activation: A user can have many profiles Activate profiles according to the situation (in-hand, on-table, in-pocket, outdoors) 87% certainty, 30 sec to calculate (not yet optimized) Context sharing (and privacy invasion): Ask to call B A Request B’s context Provide context Inform A Make decision (Voice or short message) A’s phone B’s phone Mediacup: An Active Artifact Spec: 1Mhz CPU, 15K storage, 384 byte RAM Digital temperature sensor 3-ball switch Detect movement A switch Detect whether placed on surface Infrared diode for comm. Capacitors charged wirelessly 3mm high Identified Contexts Movement context: Temperature context: Cup is stationary, drinking out of the cup, cup is played with, cup is carried around Computed by rule based heuristics with a short history of movements Filled up, cooled off, current temperature Updated every 2 seconds The cups broadcast their context to an overhead transceiver using the infrared diode Design Issues Power management System kept very low-end Motion detection uses interrupt instead of polling to enable sleep mode in 99% of the time Put the cup on the saucer to recharge wirelessly (15 minute for 12 hour) Transparency Hiding the technology does not suffice In the battery-charged prototype, users forget to replace the battery because the effect of technology is invisible! Design Issues (Cont’d) Region of impact No context-aware application in the environment Context as a common resource Inside MEETING available Outside Context-aware System Design Matthias Baldauf: context awareness1\ijahuc2007.pdf Dey:context awareness1\dey-thesis.pdf Design principles context acquisition method: predefines the CAS’s architecture to some extent. direct sensor access、middleware infrastructure、context server (Chen) Context management method for coordinating multiple processes and components (Winograd): Widgets. a widget is a software component that provides a public interface for a hardware sensor. Widgets hide lowlevel details of sensing and ease application development due to their reusability. Because of the encapsulation in widgets it is possible to exchange widgets which provide the same kind of context data (e.g., exchange a radio frequency widget by a camera widget to collect location data). Widgets are usually controlled by some kind of a widget manager. Design principles Networked services. This more flexible approach resembles the context server architecture. Instead of a global widget manager discovery techniques are used to find networked services. This service based approach is not as efficient as a widget architecture due to complex network based components but provides robustness. Blackboard model. a data-centric view (vs. processcentric); processes post messages to the blackboard, and subscribe to it to be notified when some specified event occurs. Advantages: the simplicity of adding new context sources and the easy configuration. Drawback: needs a centralized server to host the blackboard and the lack in communication efficiency as two hops per communication are needed. Layered conceptual framework application storage/management preprocessing raw data retrieval sensors Layered conceptual framework Sensor layer: physical sensors virtual sensors: source context data from software applications or services. E.g., determine an employee’s location not only by tracking, but also by browsing an electronic calendar logical sensors: combine physical and virtual sensors with additional information from various other sources in order to solve higher tasks. E.g., a logical sensor can be constructed to detect an employee’s current position by analyzing logins at desktop PCs and a database mapping of devices to location information. Layered conceptual framework Raw data retrieval layer: makes use of appropriate drivers for physical sensors and APIs for virtual and logical sensors. often implemented in reusable software components which make low-level details of hardware access transparent For instance, it is possible to replace a RFID system by a GPS system without any major modification in the current and upper layers. Layered conceptual framework Preprocessing layer: is not implemented in every context-aware system but may offer useful information if the raw data are too coarse grained. responsible for reasoning and interpreting contextual information raises the results of layer two to a higher abstraction level to meet the use of applications; E.g., the exact GPS position of a person might not be of value for an application but the name of the room the person is in, may be. Layered conceptual framework aggregation/composition of context data for several different context data sources this abstraction functionality could be implemented directly by the application. This task should better be encapsulated and moved to the context server. And by making such aggregators remotely accessible the network performance increases (as clients have to send only one request to gain highlevel data instead of connecting to various sensors) and limited client resources are saved. sensing conflicts that might occur when using several data sources. E.g., when a system is notified about a person’s location by the coordinates of her mobile phone and by a camera spotting this person. Layered conceptual framework storage and management layer: Organizes the gathered data and offers them via a public interface to the client synchronous: the client is polling the server for changes via remote method calls. Asynchronous: works via subscriptions; the client is either simply notified or a client’s method is directly involved using a call back Asynchronous approach is more suitable due to rapid changes in the underlying context. The polling technique is more resource intensive Layered conceptual framework Application layer: actual reaction on different events and context-instances is implemented here. Sometimes information retrieval and application specific context management and reasoning is encapsulated in form of agents, which communicate with the context server and act as an additional layer between the preprocessing and the application layer Design principles Context model: Key-Value、Markup scheme、Graphical、Object oriented、Logic based、Ontology based requirements for designing a context ontology: simplicity: as simple as possible to simplify the work of applications developers flexibility and extensibility: support the simple addition of new context elements and relations genericity: not be limited to special kind of context atoms but rather support different types of context Expressiveness. allow to describe as much context states as possible in arbitrary detail. Design principles Context attributes Context type Context value Time stamp: a date/time-value describing when the context was sensed. It is needed to create a context history and deal with sensing conflicts. Source: about how the information was gathered. Confidence: describes the uncertainty of this context type. Context-aware frameworks architecture, resource discovery, sensing, context model, context processing historical context data, security and privacy Architectures: context awareness1\ijahuc2007.pdf Context Managing Framework: Korpipää (2003) Service-Oriented Context-Aware Middleware (SOCAM): Gu et al. (2004a, 2004b) Context-Awareness Sub-Structure (CASS): Fahy and Clarke (2004) Context Broker Architecture (CoBrA): Chen (2003) The Context Toolkit : Salber & Dey (1999 ~2001) Hydrogen project: (Hofer et al., 2002) : context sharing CORTEX: Sentient Object Model (Biegel and Cahill, 2004) Gaia (Roman et al., 2002; Gaia Project,2005) Context-aware frameworks Resource discovery: context awareness1\ijahuc2007.pdf Context Toolkit: offers discoverer which works as registry component which interpreters, aggregators and widgets have to notify about their presence and their contact possibilities. After registration the components are pinged to ensure that they are operating. If a component does not respond to a specified number of consecutive pings, the discoverer determines that the component is unavailable and removes it from its registry list. Customers may find appropriate components querying the discoverer either via a white page lookup (a search for the components name) or a yellow page lookup (a search for specific attributes). In case the lookup was successful the discoverer returns a handle to contact the context component. SOCAM: Service Locating Service. Gaia: context providers are stored in a registry component. Hydrogen (pure peer-to-peer) only uses local built-in sensors; no discovery mechanism is involved. Context-aware frameworks sensing: context awareness1\ijahuc2007.pdf Context Toolkit: context widgets to separate applications from context acquisition concerns; permits a simple exchange of widgets since. This manner of building reusable sensor units that make the action of sensing transparent to the customer became widely accepted in distributed context-aware systems: CASS: sensor nodes SOCAM: context providers Context Managing Framework: resource servers CoBrA: context acquisition components. Context-aware frameworks Context processing: context awareness1\ijahuc2007.pdf Context Toolkit: offers facilities for both context aggregation and context interpretation. Context Managing Framework: The resource servers’ tasks are complex. First they gather raw context information from various data sources. After the preprocessing and feature abstraction, the data are delivered by posting it to the context manager’s blackboard. The context recognition services are used by the context manager to create higher-level context object out of context atoms. In this vein new recognition services are easy to add. Context-aware frameworks SOCAM: Context Reasoning Engine reasons over the knowledge base; its tasks include inferring deduced contexts, resolving context conflicts and maintaining the consistency of the context knowledge base. Different inference rules can be specified. The interpreter is implemented by using Jena2, a semantic web toolkit. CoBrA: Inference Engine processes context data. The engine contains the Context Reasoning Module responsible for aggregating context information. It reasons over the Context Knowledge Base and deduces additional knowledge from information acquired from external sources. Context-aware frameworks CASS: based on an inference engine and a knowledge base. The knowledge base contains rules queried by the inference engine to find goals using the so-called forward chaining technique. As these rules are stored in a database separated from the interpreter neither recompiling nor restarting of components is necessary when rules change. CORTEX: the whole context processing is encapsulated in Sentient Objects. The sensory capture unit performs sensor fusion to manage uncertainty of sensor data (sensing conflicts) and to build higher-level context objects. Different contexts are represented in a so-called context hierarchy together with specific actions to be undertaken in each context. Since only one context is active at any point in time (concept of the active context) the number of rules that have to be evaluated are limited. Thus efficiency of the inference process is increased. The inference engine component is based on C Language Integrated Production System (CLIPS). Context-aware frameworks Gaia: hidden in the Context Service Module allowing the creation of high-level context objects by performing first order logic operations such as quantification, implication, conjunction, disjunction, and negation of context predicates. Hydrogen: higher-level abstractions are handled by the application layer. Context-aware frameworks historical context data: context awareness1\ijahuc2007.pdf Context Toolkit, CoBrA, CASS, SOCAM and CORTEX save context data persistently in a database. A further advantage of using a database is the use of SQL which enables to read and to manipulate operations at a high abstraction level. CoBrA and CASS: the persistent storage is called Context Knowledge Base. Additionally a set of APIs is offered to assert, delete, modify and query the stored knowledge. CASS uses its database not only to save context data but also to store domain knowledge and inference rules needed for creating high-level context. Due to limited memory resources a peer-to-peer network of mobile devices like Hydrogen is not able to offer persistent storage possibilities. Context-aware frameworks Security and privacy: context awareness1\ijahuc2007.pdf Context Toolkit: introduces the concept of context ownership. Users are assigned to sensed context data as their respective owners. They are allowed to control the other users’ access. CoBrA includes an own flexible policy language to control context access, called Rei (Kagal et al., 2003). This policy language is modeled on deontic concepts of rights, prohibitions, obligations and dispensations and controls data access through dynamically modifiable domain dependent policy rules. Context-aware frameworks Summary: context awareness1\ijahuc2007.pdf Design Process of Context-aware Applications The designer does not want to worry about the middle three steps, but instead would like to concentrate on the specifying and performing the actual context-aware behaviors. 1. Specification: Specify the problem being addressed and a high-level solution. 1.1. Specify the context-aware behaviors to implement. 1.2. Determine what collection of context is required for these behaviors to be executed, using any context-acquisition mechanisms that already exist. Context Specification Need to express context about and relationships between People, Places, Things Predicates Identity (Who is…? What is…? Is with…?) Location (Near? Nearest? Distance? Path?) Activity (Is busy? Is in meeting? Current task?) Time (In past? In present? In future? On date?) Context Specification Common parameters Max number of results wanted Return name Return data type (e.g. String, List, Table) Minimal probability of correctness desired Relevant sensor input requestor has Event parameters Event rate (e.g. at most 1 event per second) Event callback (e.g. RPC, socket port) Max number of events desired Granularity of change (e.g. 1 meter) Design Process of Context-aware Applications 2. Acquisition: Determine what new hardware or sensors is needed to provide that context. 2.1. Install the sensor on the platform it requires. 2.2. Understand exactly what kind of data the sensor provides. 2.3. If no application programming interface (API) is available, write software that speaks the protocol used by the sensor. 2.4. If there is an API, learn to use the API to communicate with the sensor. 2.5. Determine how to query the sensor and how to be notified when changes occur. 2.6. Store the context. 2.7. Interpret the context, if applicable. Design Process of Context-aware Applications 3. Delivery: Provide methods to support the delivery of context to one or more, possibly remote, applications. 4. Reception: Acquire and work with the context. 4.1. Determine where the relevant sensors are and how to communicate with each other 4.2. Request and receive the context. 4.3. Convert it to a useable form through interpretation. 4.4. Analyze the information to determine usefulness. 5. Action: If context is useful, perform context-aware behavior. 5.1. Analyze the context treating it as an independent variable or by combining it with other information collected in the past or present. 5.2. Choose context-aware behavior to perform. Design Process: Delivery/Distribution Contexts typically captured remotely from applications at different time Context captured in sensor-rich environment or device may need to serve multiple applications => Need to deliver and distribute context to multiple, remote applications Infrastructure or middleware support App/network-level delivery/routing models and transport mechanism Design Process: Reception Application locates relevant sensors/contexts Service discovery Requests contexts via queries, polls, notifications Query language, event-notification mechanism How often to request? Additional interpretation/abstraction/processing Collection, aggregation, filtering, correlation, fusion,... Context Interpretation Sophisticated applications require higher level forms of context Fusion Ambiguity: Sensors not 100% reliable, e.g. confidence value Precision / Accuracy / Granularity Different ways to deal: Improve inference Probability/fuzzy model Bring the user into the loop Design Process: Action Combine received contexts with previous contexts and system/application states for further analysis Perform actions based on the analysis results May treat context collection/processing as a separate service Using the Design Process: In/Out Board 1.1: Specify the context-aware behaviors to implement. to display whether occupants of a building are in or out of the building and when they were last seen 1.2: Determine what collection of context is required and if any context-acquisition mechanisms that already exist. the relevant context types are location, identity and time; there is no existing support in the building to provide this context, so a new sensor has to be used. 2.1: Install the sensor on the platform it requires. Java iButtons® are chosen to provide the location and identity context. An iButton is a microprocessor that contains a unique identity. This identity can be read when the iButton is docked with a reader. A reader is installed on a computer, at the entrance to the research lab. Using the Design Process: In/Out Board 2.2: Understand exactly what kind of data the sensor provides. The reader combined with the iButton provides an abstract identity – rather than providing a name, which is what the developer wants, the iButton provides a 16 character hexadecimal value. 2.3/2.4: If no API is available, write software that speaks the protocol used by the sensor; If there is an API, learn to use the API to communicate with the sensor. The location context is simply the location of the installed reader, and time context is determined from the computer being used.The iButton and reader come with an API, so the developer writes supporting code that reads the abstract identity when an iButton is docked with a reader. 2.5: Determine how to query the sensor and how to be notified when changes occur. This code must support both querying and notification of events, where, informally, an event is a timely difference that makes a difference. Using the Design Process: In/Out Board 2.6: Store the context. The context information acquired from the sensor must be stored so additional code is required. 2.7: Interpret the context, if applicable. While the abstract identity does need interpretation into an actual user name, the interpretation is usually left to the application. 3: Provide methods to support the delivery of context to one or more, possibly remote, applications. design and implement a communications protocol for allowing applications to access the acquired context, including both querying and notification schemes; must be able to interact with the code that actually acquires the context. Finally, a communications mechanism must be developed to support the protocol, allowing the application to actually communicate with the sensor. Using the Design Process: In/Out Board 4.1: Determine where the relevant sensors are and how to communicate with each other The developer begins to work on the application side of the problem, determining where the sensors are and how to communicate with them. The In/Out Board needs to acquire current state information for who is in and who is out of the building, so it first locates the sensor and then queries it. It also subscribes to the iButton asking to be notified of relevant changes. 4.2: Request and receive the context. The communications is written to specifically deal with the iButton and accompanying code. Once the application receives context information (location of the reader, abstract identity and time of docking), it must convert the abstract identity to a usable form. Using the Design Process: In/Out Board 4.3/4.4: Convert it to a useable form through interpretation. It can use a simple hash table that has an associated user name for each abstract identity. After this interpretation, it analyzes the information to see if the context is useful. In this case, it means whether the iButton user was a registered user of the In/Out Board. 5.1: Analyze the context treating it as an independent variable or by combining it with other information collected in the past or present. The application must determine if the context it received was for a user arriving or leaving the building. When new useful context arrives for a given user, that user’s state is toggled from in to out, or vice-versa. 5.2: Choose context-aware behavior to perform. The developer can act on the context, and display the user’s new state on the In/Out Board. Essential and Accidental Activities Specification: essential; for without it, there can be no use of context. This step is the focal point of modeling and designing CASs, and typically impacts the rest of the application design. Acquisition: essential or accidental, depending on the sensor chosen. If the sensor has been used before accidental, reusable by the application designer. If the sensor has not been used before, then all of the difficult sub-steps must be completed. Installation of a sensor can be particularly troublesome when the platform required by the sensor does not match the platforms available or being used by the application. Storage: accidental; it is common across all sensors and is not strictly dependent on the sensor being used. Essential and Accidental Activities Delivery: The delivery of context is an accidental step. The designer merely wants to acquire the context and should not be concerned about these types of low-level details. Reception: accidental; the designer most likely does not care about the details of context reception and determining usefulness, and is concerned only with receiving useful context. Action: The analysis from analyzing the types and values of context received is accidental and should be automatically provided by the supporting infrastructure. The determination of actions is application-specific and is essential. Revised Design Process Specification: Specify the problem being addressed and a high-level solution. 1.1. Specify the context-aware behaviors to implement. 1.2. Determine what context is required for these behaviors (with a knowledge of what is available from the environment) and request it. Acquisition: Determine what hardware or sensors are available to provide that context and install them. 3. Action: Choose and perform context-aware behavior. Revised Design Process If all necessary context acquisition components are already available for use by applications, then step 2 is not required. To use this reduced design process. we must provide infrastructure level support. Connecting application design too tightly to sensors used has resulted from the use of the longer, more complex design process. 3 basic problems emerged due to this trend: the lack of variety in sensors used, the lack of variety in the types of context used and the inability to evolve context-aware applications. required features for a supporting framework library programmers, toolkit programmers (may add something new to the architecture) context specification: need a mechanism (& specification language) that allows application builders to specify what context an application requires along a number of dimensions: single piece of context vs. multiple pieces of context. if multiple, related context vs. unrelated context. unfiltered context vs. filtered context. uninterpreted context vs. interpreted context. required features for a supporting framework the mechanism must support multiple specifications from multiple applications that may be requesting the same or different set of context. This mechanism makes it easier for application designers to specify what context their applications require. It helps remove the accidental design process steps of reception (receiving the context, interpretation and analysis) and the first step of action (further analysis with additional context to determine usefulness). required features for a supporting framework Separation of Concerns and Context Handling there is no common way to acquire and handle context. Application developers choose whichever technique is easiest to implement, at the expense of generality and reuse. common ways in which context is handled: connecting sensor drivers directly into applications and using servers to hide sensor details applications are difficult to build and evolve. required features for a supporting framework handle context in the same manner as we handle user input. User interface toolkits provide an important abstraction to enable designers to use input without worrying about how the input was collected. This abstraction is called a widget, or an interactor. By separating how context is acquired from how it is used, the acquisition step is able to greatly reduced. Applications can now use contextual information without worrying about the details of a sensor and how to acquire context from it. required features for a supporting framework Context Interpretation There may be multiple layers that context data go through before reaching an application, due to the need for additional abstraction. The use of these multiple layers should be transparent. In order to support this transparency, context must often be interpreted before it can be used by an application. E.g., an application wants to be notified when meetings occur. At the lowest level, location information can be interpreted to determine where various users are and identity information is used to check co-location. At the next level, this information could be combined with sound level information to determine if a meeting is taking place. required features for a supporting framework In order for the interpretation to be easily reusable by multiple applications, it needs to be provided by the framework. Otherwise, each application would have to re-implement the necessary implementation. The interpretation mechanism removes the interpretation sub-steps found in both the acquisition and reception steps of the design process. Combined with the specification mechanism, it allows applications to simply request the context they want, regardless of whether interpretation is required or not, and for the applications to receive the requested context. required features for a supporting framework Transparent Distributed Communications As environments and computers are becoming more instrumented, more context can be sensed, but this context will be coming from multiple, distributed machines connected via a computer network. The fact that communication is distributed should be transparent to both sensors and applications. This simplifies the design and building of both sensors and applications, relieving the designer of having to build a communications framework. Without it, the designer would have to design and implement a communications protocol and design and implement an encoding scheme (and accompanying decoder) for passing context information. required features for a supporting framework A related requirement is the need for a global timeclock mechanism. Transparent communication allows the delivery step of the design process to be removed completely along with the first step of the reception step, and is an integral part of the context specification mechanism. The provision of a global timeclock makes complex analysis of context possible by allowing context data to be compared on a temporal basis. required features for a supporting framework Constant Availability of Context Acquisition CASs should not instantiate individual sensor components, but must be able to access existing ones. Furthermore, multiple applications may need to access the same piece of context. requirement that the components that acquire context must be executing independently from the applications that use them. This eases the programming burden on the application designer by not requiring her to instantiate, maintain or keep track of components that acquire context, while allowing her to easily communicate with them, as needed. required features for a supporting framework Because these components run independently of applications, there is a need for them to be persistent, available all the time. Constant availability of context acquisition components facilitates the ability to locate existing context sensing components found in the specification step. Because these components are persistent and are independent from applications, they can be found and used by any application. If existing components can be found, the need for the acquisition step is removed. required features for a supporting framework Context Storage A requirement linked to the need for constant availability is the desire to maintain historical information. A component that acquires context information, should maintain a history of all the context it obtains. this feature removes the substep dealing with the storing of context in the acquisition step of the design process. It also allows complex analysis to be performed by providing a source of historical/past context. required features for a supporting framework Resource Discovery With a resource discovery mechanism, when an application is started, it could specify the type of context information required. The mechanism would be responsible for finding any applicable components and for providing the application with ways to access them. For example, in the In/Out Board, rather than hard-coding the location of the iButton reader being used, the developer can indicate that the application is to be notified whenever any user of the In/Out Board enters or leaves the building. required features for a supporting framework This resource discovery feature would not only be useful at runtime, but also at design-time. An application designer could use the resource discovery mechanism to determine whether the context she requires is already available in the environment or in combination with the specification mechanism and the constant availability of context components to determine what situations can be sensed and whether a given context request can be fulfilled by the currently executing infrastructure. If the current infrastructure does not provide the needed context, the designer will know what components need to be added. required features for a supporting framework A resource discovery feature allows us to remove the first part of the reception step in the design process. With this feature, applications do not need to explicitly or a priori know where specific sensors are. Instead, in combination with the specification mechanism, they can request a certain type of context and be automatically connected to the sensor or interpreter that can satisfy the request. at design time, a developer can locate what components already exist in the environment, allowing her to determine what components, if any, need to be added, simplifying the specification step. System Issues (1/2) Programming model Programming the physical world Unreliable sensors, recognition algorithms, plus standard distributed computing issues Interoperability Sensors, services, and devices Useless if everyone has proprietary / custom systems Need standard data formats, protocols, and frameworks Varying capabilities of sensors, services, and devices Evaluation System Issues (2/2) May need a middleware layer to decouple applications and context sensing Collect raw context, translate to application-understandable format, disseminate it Centralized context server Distributed architecture Intelligence Who is smart? User or system or both Who makes the decisions on what actions to take? Tradeoff between user cognitive load and effort to make system “smart” People Issues Avoiding embarrassing situations Active Badges + bathrooms Inconvenient phone forwarding Avoiding dangerous situations Need to take into consideration cost of mistake Smoke alarms when cooking Lights that turn off when you're still there Woman locked in "smart toilet stall" Will adding more context really help here? People Issues Making it predictable and understandable Setting preferences "I want my cell phone to ring except in theaters and when I'm in a meeting unless…" Why the heck did it do that? Privacy What does the computer know about me? What do others know about me? Capturing/collecting lots of information about people, places and devices People uncomfortable when don’t know what is being collected and how it’s used Killer Applications? Need something to focus and drive the research Need something to put in the hands of real people Business model: how to make money from it? Middleware for Distributed Context-aware Applications Henricksen:context awareness1\middleware distributed context-aware systems.pdf Dey:context awareness1\chi99.pdf Dey:context awareness1\Toolkit.pdf Dey:context awareness1\dey-thesis.pdf CAS structure and middleware requirements Figure 2: Henricksen:context awareness1\middleware distributed context-aware systems.pdf Requirements for middleware for context-aware systems Support for heterogeneity: Hardware components ranging from resource-poor sensors, actuators and mobile client devices to high-performance servers must be supported, as must a variety of networking interfaces and programming languages. Legacy components may be present. Support for mobility: All components (especially sensors and applications) can be mobile, and the communication protocols that underpin the system must therefore support appropriately flexible forms of routing. Context information may need to migrate with context-aware components. Flexible component discovery mechanisms are required. CAS structure and middleware requirements Scalability: Context processing components and communication protocols must perform adequately in systems ranging from few to many sensors, actuators and application components. Similarly, they must scale to many administrative domains. Support for privacy: Flows of context information between the distributed components of a context-aware system must be controlled according to users’ privacy needs and expectations. CAS structure and middleware requirements Traceability and control: The state of the system components and information flows between components should be open to inspection - and, where relevant, manipulation - in order to provide adequate understanding and control of the system to users, and to facilitate debugging. Tolerance for component failures: Sensors and other components are likely to fail in the ordinary operation of a context-aware system. Disconnections may also occur. The system must continue operation, without requiring excessive resources to detect and handle failures. Ease of deployment & configuration: The distributed hardware and software components of a context-aware system must be easily deployed and configured to meet userand environmental requirements, potentially by nonexperts (for example, in “smart home” environments). PACE middleware design principles: The context model used in a CAS should be explicitly represented within the system. This representation should be separate from the application components (layer 4) and the parts of the system concerned with sensing and actuation (layers 0 and 1), so that the context model can evolve independently, without requiring any components to be re-implemented. The context-aware behavior of CAAs should be determined, at least in part, by external specifications that can be customized by users and evolved along with the context model (again, without forcing re-implementation of any components). The communication between application components, and between the components and middleware services, should not be tightly bound to the application logic, so that a significant reimplementation effort is required when the underlying transport protocols or service interfaces change. Structure of PACE a context management system (layer 2); a preference management system that provides customisable decision-support for context-aware applications (layer 3); a programming toolkit that facilitates interaction between application components and the context and preference management systems (layer 4); tools that assist with generating components that can be used by all layers, including a flexible messaging framework. Context Management System fulfils the requirements of layer 2: provides aggregation and storage of CI, in addition to performing query evaluation. uses a two-layered context modelling approach: context can be expressed both in terms of fine-grained facts and higher-level situations which capture logical conditions that can be true, false or unknown in a certain context. All information is stored in the fact representation, but can be queried by either retrieving specific facts based on template matching, or evaluating situation definitions over a set of facts. consists of a distributed set of context repositories. Each repository manages a catalog, which is a collection of context models consisting of fact type and situation definitions. Context Management System Applications may define their own context models or share them with other applications. Context-aware components are not statically linked to a single repository, but can discover repositories dynamically by catalog name (and potentially also other attributes). Several methods of interacting with a context repository are currently permitted, in order to support a range of client programming languages and platforms A variety of discovery mechanisms can be used, including context-based discovery, which allows for matching based on context attributes. Context Management System Each repository is capable of performing access control, although this feature can be switched off if it is not required. The access control mechanism allows users to define privacy preferences that dictate the circumstances (i.e.,situations) in which context information can be queried and updated. The privacy preferences are stored and evaluated by the preference management system. Context Management System running on top of a relational database management system. written in Java using JDBC to query and manipulate a set of context databases. It provides clients with the following interfaces: query: supports situation evaluation and retrieval of facts matching supplied templates update: allows insertion, deletion and modification of facts, as well as insertion of new situation definitions transaction: allows clients to create read-only transactions within which a sequence of queries can be executed against a consistent set of context information, regardless of concurrent updates; subscription: allows monitoring of situations and fact types, using callbacks to notification interfaces implemented by clients; and metadata: allows clients to discover the fact types and situations that are defined by models in the catalog. Context Management System In addition to invoking methods on repositories using Java RMI, clients can use a Web interface (based on XML and HTTP) or programming language stubs generated from a context model specification. The latter method can potentially accommodate arbitrary programming languages and communication protocols; currently, we generate stubs for Java and Python, using Elvin, a content-based message routing scheme, as the underlying communication paradigm. Context Management System One of the benefits of Elvin is that it allows for complex interactions (including 1:N and N:M communication, not only 1:1 as supported by RMI and HTTP), which allows (for example) queries and updates to be simultaneously routed to multiple context repositories a model is being developed for replicating context information across several repositories and allowing clients to cache their own context information for use during disconnections. Preference Management System provides layer 3 functionality that builds on functionality of the context management system. assists context-aware applications with making context-based decisions on behalf of users. Its main roles are to provide storage of user preference information and evaluation of preferences - with respect to application state variables and context information stored by the context management system - to determine which application actions are preferred by the user in the current context. Applications can connect to, and store their preference information in, one or more preference repositories. Preference Management System The preferences are defined in terms of a novel preference model, which allows the description of contextdependent requirements in a form that enables them to be combined on-the-fly to support decisions about users’ preferred choice(s) from a set of available candidates. E. g., the preference model can be used to decide which mode of input or output should be employed for particular users according to their requirements and current contexts. benefits to decision-making: customisation and evolution of context-aware behaviour can be supported in a straightforward manner; preferences can be shared and exchanged between applications; new types of context information can be incorporated into decision-making processes simply by adding new preferences, without the need to modify the application components. Preference Management System Its implementation bears strong resembles to that of the context management system It provides the following interfaces: update: allows new preferences to be defined and grouped appropriately into sets (for instance, by owner and purpose); query: provides preference evaluation based on the information stored in the context management system; transaction: allows a set of preference evaluations to occur over a consistent set of context information, regardless of concurrent updates occurring within the context management layer; metadata: allows retrieval of preference and preference set definitions. similar to the context repositories, the preference repositories respond to requests from clients over a variety of communication protocols. However, Java clients need not interact directly with repositories; instead, they are provided with a Java programming toolkit that assists with discovery of, and interaction with, repositories. Programming Toolkit complements the functionality of the preference management layer by implementing a simple conceptual model for formulating and carrying out context-based choices. The model provides a mechanism for linking application actions with candidate choices. also allows one or more of the actions to be automatically invoked on the basis of the results of evaluating the choices with respect to preference and context information, using the services of the preference and context management systems. Programming Toolkit benefit: makes the process of discovering and communicating with the preference and context management systems transparent to applications. helps to produce applications that are cleanly structured and decoupled from their context models, and thus better able to support changes in the available context information. These changes can result from evolution of the sensing infrastructure over time, or problems such as disconnection or migration from a sensor-rich environment to a sensor-poor one. implemented in Java, using RMI for communication with remote components; however, it could be ported to other programming languages and communication protocols in the future. Messaging Framework To facilitate remote communication between components of context-aware systems In the tradition of middleware such as CORBA, the framework aims to provide various forms of transparency, such as location and migration transparency. It maps interface definitions to communication stubs that are appropriate for the deployment environment. These stubs are considerably simpler for the programmer to work with than the APIs of the underlying transport layers, and can also be automatically re-generated at a later date, allowing for substitution of transport layers without modifying the application. Messaging Framework Stubs can be generated for a variety of programming languages and communication protocols (including message-based and RPC-based protocols). To date, focused on producing Java and Python stubs for the Elvin publish/subscribe content-based message routing scheme. Elvin is particularly appropriate for building context-aware systems because it decouples communication from cooperation. Messaging Framework Because it delivers messages based on matches between message content and the subscriptions of listeners, rather than based on explicit addressing, it is able to tolerate mobility, support complex interactions (not only 1:1 interactions as in the case of RPC/RMI), and allow for spontaneous interactions between components without the need for an explicit discovery/configuration step. The ability to add new listeners into the system on-the-fly is also useful for debugging and generating traces. Schema Compiler Toolset produce custom components to assist with developing and deploying context-aware systems. The tools take input in the form of a textual representation of a context model (a context schema), perform checks to verify the integrity of the model, and produce the following outputs: SQL scripts to load and remove context model definitions from the relational databases used by our context repositories; model-specific helper classes to simplify source code concerned with carrying out context queries and updates; context model interface definitions compatible with the messaging framework. Schema Compiler Toolset The first output simplifies the deployment and evolution of context models. By automating the mapping of context models into the database structures stored by the context repositories, errors that might arise during the handcoding of SQL scripts or JDBC code to manipulate the repositories can be avoided. updates to context models can be supported simply by re-generating and re-executing the scripts. In the future, we envision extending the tools to produce alternative scripts for context repositories that are not SQL-based. Schema Compiler Toolset The second output is designed to simplify the programming of components that query or update a context model, and includes classes that represent basic value types, fact types and situations defined by the model. By programming with these classes, rather than the generic APIs provided by the context management layer, type checking becomes possible at compile time and standard IDE features such as code completion can better be exploited. Schema Compiler Toolset The final output is used to produce stubs for transmitting/receiving context information over communications infrastructure such as Elvin. The context transmitters can be used by layer 0 and layer 1 components (sensors, actuators and processing components) to transmit context information to one or more context repositories. Similarly, the context receivers can be used at layer 2 to listen for context updates that require mapping to operations on context repositories. Case Study: Vertical Handover adapting the streaming of media to a mobile user according to the context. The adaptation occurs at the application layer rather than the network layer (e.g., using Mobile IP) because of stringent Quality of Service (QoS) requirements. Handover occurs in response to any context change: e.g., the user moving into range of a network that offers higher bandwidth than the current network, or the signal strength of the current network dropping. The handover process is managed by adaptation managers and proxies. Case Study: Vertical Handover The adaptation managers use the context management system to monitor significant context changes, in order to determine when a vertical handover should occur, and to which network interface. The proxies perform the handover process. One proxy is co-located with the transmitter, while other proxies are located within the same networks as the receivers. The transmitter’s local proxy (proxy-transmitter) is responsible for redirecting the stream when it receives a handover instruction from an adaptation manager. Case Study: Vertical Handover During the handover process, the proxy-transmitter transmits the stream to both the original and the new proxy; referred to as doublecasting. The proxies within the receivers’ networks are responsible for forwarding the streams to the receiver(s) executing on the client device(s). When the handover is complete, the proxytransmitter stops transmitting to the original receiver proxy. he application adapts by handing Implementation of Case Study Architecture (Fig. 2), context model (Fig. 3): context awareness1\middleware distributed context-aware systems.pdf. Model: main objects: computing devices, networks, network interfaces, streams and proxies. captures associations between computing devices and network interfaces, proximity between devices, mappings of streams to proxies and network interfaces, basic QoS information related to network interfaces (current signal strength and bandwidth), and other type and configuration information. Implementation of Case Study Much of the information is user-or application-supplied (i.e., static or profiled in the terminology of our context modelling approach); however, proximity between devices is sensed using wireless beacons, and current network connectivity, signal strength and bandwidth are all sensed by monitors running in the network. The context model, and its instantiation at run-time with concrete facts, is managed by a set of context repositories as shown in Fig. 2. Each local network may contain one or more repositories. The schema compiler toolset was used to map the context model to appropriate database structures when the context repositories were deployed. Implementation of Case Study The toolset was also used to generate context transmitter and receiver stubs, which are used by the wireless beacons (not shown in the diagram), network monitors and adaptation managers (shown for networks 1 and 3) to report context information to the context repositories over Elvin, via context listeners that map the Elvin notifications to RMI context repository updates. Adaptation managers: The context-aware functionality is concentrated within the adaptation managers. Implementation of Case Study They are the components that are responsible for determining when handover is required, according to the current context and user preferences. the adaptation managers are the components that interact with the context and preference repositories. the adaptation managers are not tightly coupled to the context model. The only direct interaction that occurs between the adaptation managers and the context repositories is in the form of subscriptions/notifications, which allow the managers to learn about significant context changes and report new streaming configurations. Implementation of Case Study The subscriptions monitor the state of the sensed fact types. When an adaptation manager is notified of a change (for example, a drop in signal strength, as shown in Fig. 2), it uses the programming toolkit to connect to a preference repository, re-evaluate the user’s preferences, and determine whether the current network interface is still the preferred one (step 4 in the figure). The bulk of the context evaluation occurs during this step, as a side-effect of the preference evaluation (step 4.1). Implementation of Case Study New context information can be easily incorporated into the evaluation simply by extending the user preferences (i.e., the implementation of the adaptation manager does not need to change). When an adaptation manager determines that a handover is required, it communicates with the proxies (step 5) and the context listeners (step 6) using Elvin. The manager first transmits a handover instruction to the proxies using Elvin stubs produced by the PACE messaging framework. Implementation of Case Study After instructing the proxies to perform the handover, the adaptation manager updates the stream state information stored in the context repositories (i.e., the “streaming...to” fact type shown in Fig. 3), using a context transmitter stub to transmit the information to the context listeners. Analysis of Case Study Heterogeneity: PACE messaging framework can generate stubs for a variety of programming languages (and, in the future, transport layers). capable of accommodating legacy components, such as the transmitters and receivers in the vertical handover system. Mobility: The use of Elvin within the messaging framework facilitates component mobility, and often removes the need for component discovery. Local context and preference repositories can be dynamically discovered by mobile context-aware components using a variety of service discovery protocols. Analysis of Case Study In the future, PACE will be extended to support for mobility by introducing aching/hoarding models for context and preference information, to allow mobile components to store local copies of information that is relevant to users. Scalability: current version does not address scalability or performance the same can be said of almost all of existing solutions As future work, we intend to develop models for federating context and preference managers across large scale systems and a large number of administrative domains. Analysis of Case Study Privacy: (few solutions address this issue) provide access control for sensitive context information. intend to add access control to our preference management system and combine this access control with context-based authentication. Traceability and Control: (few address this issue) The use of Elvin facilitates the generation of traces, as event listeners can be added on-the-fly and event traces can be tailored by adjusting the Elvin subscriptions. the preference model also provides a basic mechanism for user control and customization. In the future, we envisage opening up the service layers to clients to allow inspection (and manipulation) of context and preference evaluations. Traces of these evaluations can be selectively revealed to users to explain system behaviors. Analysis of Case Study In the future, we envisage opening up the service layers to clients to allow inspection (and manipulation) of context and preference evaluations. Traces of these evaluations can be selectively revealed to users to explain system behaviors. Tolerance for failures: ranks behind that of Solar but ahead of the remaining solutions surveyed. Although our middleware does not yet detect or repair failed components, its use of Elvin allows a loose coupling of components, minimizing the impact of disconnections and failures. our context and preference models were both designed with the assumption that context information will generally be imperfect. This introduces some tolerance for failed sensors, sensing errors, and so on. Analysis of Case Study Deployment and Configuration: PACE provides more advanced support for component deployment and configuration than previous solutions. the messaging framework simplifies the deployment of components on top of a variety of platforms, while the schema compiler toolset facilitates the deployment of new context models. further extensions to the middleware are needed to facilitate the scalable deployment and configuration of infrastructural components such as sensors. Context Toolkit “The Context Toolkit: Aiding the Development of Context-Enabled Applications” D. Salber, A.K. Dey, G.D. Abowd; Georgia Institute of Technology, Proc. Of CHI’99 Dey:context awareness1\chi99.pdf Dey:context awareness1\Toolkit.pdf Dey:context awareness1\dey-thesis.pdf Difficulties in Using Context It is acquired from unconventional sensors. Mobile devices for instance may acquire location information from outdoor GPS receivers or indoor positioning systems. Tracking the location of people or detecting their presence may require Active Badge devices, floorembedded presence sensors or video image processing. It must be abstracted to make sense for the application. GPS receivers for instance provide geographical coordinates. But tour guide applications would make better use of higher-level information such as street or building names. Similarly, Active Badges provide IDs, which must be abstracted into user names and Locations. Difficulties in Using Context It may be acquired from multiple distributed and heterogeneous sources. Tracking the location of users in an office requires gathering information from multiple sensors throughout the office. Furthermore, context sensing technologies such as video image processing may introduce uncertainty: they usually provide a ranked list of candidate results. Detecting the presence of people in a room reliably may require combining the results of several techniques such as image processing, audio processing, floor-embedded pressure sensors, etc. Difficulties in Using Context It is dynamic. Changes in the environment must be detected in real time and applications must adapt to constant changes. For instance, when a user equipped with a handheld moves away from the electronic whiteboard, the user loses the benefit of the wide display surface and the application must modify its behavior accordingly. Also, context information history is valuable, as shown by context-based retrieval applications. A dynamic and historical model allows applications to fully exploit the richness of context information. => Lack conceptual model and tools to solve => Learning from GUI => context widgets Difficulties in Using Context It is dynamic. Changes in the environment must be detected in real time and applications must adapt to constant changes. For instance, when a user equipped with a handheld moves away from the electronic whiteboard, the user loses the benefit of the wide display surface and the application must modify its behavior accordingly. Also, context information history is valuable, as shown by context-based retrieval applications. A dynamic and historical model allows applications to fully exploit the richness of context information. => Lack conceptual model and tools to solve => Learning from GUI => context widgets GUI Toolkits GUI widgets (also called interactors) span a large range of interaction solutions: selecting a file; triggering an action; choosing options; or even direct manipulation of graphical objects. Benefits They hide specifics of physical interaction devices from the applications programmer so that those devices can change with minimal impact on applications. Whether the user points and clicks with a mouse or fingers and taps on a touchpad or uses keyboard shortcuts doesn’t require any changes to the application. GUI Toolkits They manage the details of the interaction to provide applications with relevant results of user actions. Widget-specific dialogue is handled by the widget itself, and the application often only needs to implement a single callback to be notified of the result of an interaction sequence. They provide reusable building blocks of presentation to be defined once and reused, combined, and/or tailored for use in many applications. Widgets provide encapsulation of appearance and behavior. The programmer doesn’t need to know the inner workings of a widget to use it. GUI Toolkits Limitations: too low-level or lacking flexibility provide stepping stones for designing and building user interfaces and developing tools such as User Interface Management Systems (UIMS). Context Toolkit Toolkit for distributed context-aware applications makes the distributed nature of context transparent to context-aware applications. Framework for acquiring and handling context Standard components Applications do not need to know whether these context components are being executed remotely or locally. All components share a common communications mechanism (XML over HTTP) that supports the transparent distribution. These components run independently of any single application, allowing them to be used by multiple applications. Context Toolkit Dey:context awareness1\Toolkit.pdf Dey:context awareness1\dey-thesis.pdf BaseObject: a basic communications component It facilitates the peer-to-peer communications used in the toolkit, being able to send, receive, initiate and respond to communications. Applications use BaseObject instances to communicate with the context-sensing infrastructure and components within the infrastructure sub-class from BaseObject to allow them to communicate with other infrastructure components. Context Toolkit key abstractions (Fig. 7, Fig. 8) Widgets (Services) Aggregators Interpreters Discoverer Context Interpreters Convert or interpret context to higher level information App Location to Room Interpreter Location Widget Location to Street Interpreter Context Aggregators Collect contexts relevant to particular entities Further separation, simplifies design App App Location to Room Interpreter Person Aggregator Location Widget Activity Widget Affect Widget Context Services Perform behaviors that act on the environment In/out board Context Services A context service is an analog to the context widget. Whereas the context widget is responsible for retrieving state information about the environment from a sensor (i.e. input), the context service is responsible for controlling or changing state information in the environment using an actuator (i.e. output). Context services can be synchronous or asynchronous. Ex. of Synchronous: send an e-mail to a user. Ex. of asynchronous: send a message to a user on a twoway pager containing a number of possible message responses. The service is complete when the user chooses a response and returns it. The service probably does not want to wait indefinitely for a response. Context Services To build a context service, a developer does not need to worry about providing a mechanism for calling and executing the service, dealing with transparent distributed communications, separation from applications, and allowing applications to discover it, but instead receives this functionality automatically. Instead, the developer must provide a description of the service’s functionality, the set of parameters, if any, that the service requires to be executed (e.g. the message and potential responses to page a user with, the user’s pager number), and code that interacts with an actuator to perform the service. Context Discoverer Registry for context components In/out board Context Widget A software component that provides applications with access to context information from their operating environment insulates applications from context acquisition Analogy to GUI widget Separation, callbacks, attributes, encapsulation, abstraction e.g. GUI button Why: Responsible for acquiring and abstracting data from particular sensor, separation of concerns, storage Context Widget benefits: They hide the complexity of the actual sensors used from the application. Whether the presence of people is sensed using Active Badges, floor sensors, video image processing or a combination of these should not impact the application. They abstract context information to suit the expected needs of applications. A widget that tracks the location of a user within a building or a city notifies the application only when the user moves from one room to another, or from one street corner to another, and doesn’t report less significant moves to the application. Widgets provide abstracted information that we expect applications to need the most frequently. Context Widget They provide reusable and customizable building blocks of context sensing. A widget that tracks the location of a user can be used by a variety of applications, from tour guides to office awareness systems. Furthermore, context widgets can be tailored and combined in ways similar to GUI widgets. For example, a Presence widget senses the presence of people in a room. A Meeting widget may rely on a Presence widget and assume a meeting is beginning when two or more people are present. These benefits address issues 1 and 2 due to the characteristics of context and issues 3 and 4, distribution and dynamicity, the context toolkit has some unique features. Context Widgets Widgets abstract out sensors App App Location Widget Active Badge Active Badge GPS Cell Phone Location Context Widgets Context widgets have a state and a behavior State: a set of attributes that can be queries or subscribed (via widget triggers) by applications E.g., an IdentityPresence widget has attributes for its location, the last time a presence was detected, and the identity of the last user detected. Applications can also register to be notified of context changes detected by the widget. The widget triggers callbacks to the application when changes in the environment are detected. The IdentityPresence widget for instance, provides callbacks to notify the application when a new person arrives, or when a person leaves. Context Widgets basic building blocks that manage sensing of context. widget toolkit provides means of composing widgets: to take advantage of multiple types of context information. For example, a widget designed to detect the kind of activity people in a classroom are engaged in could combine the information provided by presence widgets and activity sensing widgets using, for instance, audio and video analysis. Based on information provided by widgets such as the number of people in the room, their location in the room, the speakers, activity in the front of the classroom, the composite widget would detect activities such as lecture, group study, exam, etc. Context Widgets Differences from GIU widgets: live in a distributed architecture: rely on three kinds of distributed components: generators that acquire context information, interpreters that abstract it and servers that aggregate information. Applications, widgets and the components they rely upon may be distributed. (address issue 3) monitor environmental information that may be needed at any time by an application. Thus a context widget is active all the time, and its activation is not, as with GUI widgets, driven by applications. (address issue 4) Example Context Widgets (I) IdentityPresence: senses the presence of people and their identity Generators Acquires raw information from the sensor(s) Could be voice recognition, Active Badges, video/image recognition, keyboard and login information Attributes and callbacks are independent of generators Context Widgets All widgets acquire context information through generators. Generators are components that encapsulate a single sensor or a set of closely related sensors and the software that acquires raw information from the sensor(s). The IdentityPresence widget could be implemented using any number of generators, including voice recognition, Active Badges, video/image recognition, keyboard and login information, or even a combination of these. Context Widgets The generator that is chosen affects neither the definition of the widget nor any application that uses the widget. The attributes and callbacks provided by the widget are independent from the actual implementation, thus sheltering the application from the specifics of the sensors used. Our current implementation of the IdentityPresence widget uses Dallas Semiconductor’s i Buttons, passive tags with unique identifiers and storage and computing capabilities or alternatively passive TIRIS RF tags. Example Context Widgets (II) Activity: senses the current activity level at a location Generators: Microphone, infrared sensor, video image analysis Other Context Widgets NamePresence: PhoneUse: Whether a phone is being used and the length of use MachineUse: Provides the user’s actual name When a user logs onto or off of a computer, his identity, and length of her computing session GroupURLPresence: Provides a URL relevant to the research group a user belongs to when her presence is detected Example of Using Context Widgets Dey:context awareness1\chi99.pdf In/out board: One IdentityPresence at entrance to building Example of Using Context Widgets Information Display: Displays information relevant to the user’s location and identity on a display adjacent to the user Activates itself as someone approaches it, and the information it displays changes to match the user, her research group, and location. Context : location of the display, the identity of the user, the research group the user belongs to and information that is interesting to that research group GroupURLPresence or IdentityPresence Example of Using Context Widgets DUMMBO (Dynamic Ubiquitous Mobile Meeting Board) Digitizing whiteboard to capture and access informal and spontaneous meetings Capture ink written to and erased from whiteboard as well as the recorded audio discussion Activated when two or more people gathered around Context: participants’ identities, time of arrival at or depart whiteboard, location of whiteboard Multiple NamePresence, one for each location where DUMMBO could be moved to, one on DUMMBO itself Context Implementation Details Dey:context awareness1\chi99.pdf Distribution: The context infrastructure must accommodate distribution of applications, widgets and the components they rely upon, across a network. Applications may require the services of several widgets distributed across different machines, as described in the DUMMBO application. widgets themselves may be distributed. A widget may consist of any number of three types of components: generators, interpreters, and servers (see figure 2). Context Implementation Details An interpreter abstracts raw or low-level context information into higher level information. An example of this was seen in the DUMMBO application where the basic NamePresence widget used a generator to obtain a user ID for the user whose presence was detected. An interpreter is used to abstract the raw ID information into an actual user name. Interpreters can be used for more than simply converting between types of data. They play an important role in widget composition. Context Implementation Details Handle composition:context awareness1\chi99.pdf e.g., IdentityPresence + Activity Meeting Interpreters can be used to assess validity of uncertain information provided by multiple generators Server: a widget that collects, stores and interprets information from other widgets Often used to model context of real world entities, e.g., users or places, from elementary widgets May include privacy handling Context Implementation Details Handle communicating across heterogeneous components: Assume that the underlying system supports TCP/IP Communication model uses the HTTP protocol Language model uses the ASCII-based Extensible Markup Language (XML) Context Implementation Details Handle dynamism: Environment changes application adapt Need: to access information and to access only desired information, also need to access context history Use a subscription mechanism to notify an application of context changes and a polling mechanism to allow an application to inquire Allow conditions to be specified before the widgets will be notified filtering unwanted info at infrastructure All widgets store historical data in a database for applications or interpreters to retrieve Implementation Details (BaseObject) the basic communications infrastructure of the Context Toolkit Contains methods for communicating with widgets, aggregators, interpreters, services and discoverers. For the outgoing communications, where BaseObject behaves as a client, data is first encoded using XMLand then wrapped with HTTP To communicate with any context component, three parameters must be provided: hostname (or I.P. address) of the machine the component is executing on, the port number the component is listening for communications on and the unique identity of the component. Implementation Details (BaseObject) These parameters do not have to be known a priori by the object using a BaseObject instance, but can be determined at runtime using a Discoverer. There are some methods that BaseObject supports for communicating with all types of components in the infrastructure. This includes a method that “pings” another component to ensure that it is still operating and one that queries for the version of a remote component. Client: pingComponent: ping a component queryVersion: request a component’s version Server: pingComponentReply: response to a ping queryVersionReply: response containing version number Implementation Details (BaseObject) When any component receives a communications message, it behaves like a server To handle simultaneous incoming communications, BaseObject uses a multithreaded server. BaseObject is responsible for maintaining a global timeclock. Each instance automatically connects to a Simple Network Time Protocol (SNTP) server in order to synchronize its local time with a standard global time. Synchronization occurs at regular intervals to ensure that all components and applications have a common time with which to compare data. Implementation Details (Widgets) A basic Widget object handling all the features that are common across widgets. It is responsible for maintaining a list of subscribers to each callback in the widget, storing all data that is sensed, mediating access to the widget’s services and allowing inspection of the widget’s internal information. able to act as a client or a server. Most commonly, widgets behave as servers that deliver requested information to clients. When an incoming message is received by a widget, BaseObject tries to handle it. If BaseObject is unable to handle it, the widget handles the message and responds. Implementation Details (Widgets) Widgets can handle the following types of messages or requests:context awareness1\dey-thesis.pdf Widget inspection queryAttributes: Request for a list of the context attributes the widget can provide queryCallbacks: Request for a list of the callbacks the widget can provide notification about queryServices: Request for a list of the services a widget can perform Widget subscriptions: (Fig. 14, 16) addSubscriber: Request to subscribe to a widget callback removeSubscriber: Request to unsubscribe to a widget callback Implementation Details (Widgets) Widget storage query: Query the widget for its last acquired value updateAndQuery: Request that the widget acquire new data and return it retrieveData: Request to retrieve data from the widget’s persistent storage Widgets have two mechanisms for storing data. The simplest is a local cache in memory that stores only recently sensed context. Context data is flushed from the local cache to the second, and more persistent, store according to a specified flush policy. An application can acquire this historical context from the persistent store using a retrieveData request. Implementation Details (Widgets) Widget services (Fig. 17,19) serviceRequest: Request to execute a widget’s asynchronous or synchronous service context awareness1\dey-thesis.pdf Implementation Details (Discoverer) the application uses the Discoverer to locate any context that can provide information it is responsible for It specifies the attributes it is interested in, The Discoverer uses this information to locate any widgets and aggregators that can return this information, if any. It returns all the information it has about each relevant component, including the hostname or I.P. address of the computer the component is executing on, the port number the component receives communications on and the component’s unique identity. The application also subscribes to the Discoverer to be notified about any changes to the components it is using or the appearance of a new component that can meet the specified conditions. Implementation Details (Discoverer) If no component is available, an application-specific decision must be made as to what to do. programmers can use a Discoverer to determine what context can be currently sensed from the environment. The Context Toolkit allows for multiple Discoverers. Each Discoverer is typically responsible for all the components in a physical environment. Components are not limited to registering with a single Discoverer and can register with multiple Discoverers, if desired. To connect the information held by multiple distributed Discoverers, each of these Discoverers can register with a higher-level Discoverer, forming a scalable hierarchy or federation of Discoverer objects. Implementation Details (Discoverer) Discoverers can handle the following types of messages or requests: Discoverer registration Discoverer subscriptions addDiscoveredObject: Register a component with the Discoverer removeDiscoveredObject: Unregister or remove a component from the Discoverer subscribeToDiscoveredObjects: Subscribe to the Discoverer for discovered objects removeDiscovererSubscriber: Unsubscribe from the Discoverer Discoverer queries retrieveKnownDiscoveredObjects: Query the Discoverer for its discovered objects Implementation Details (Discoverer) Fig. 21: context awareness1\dey-thesis.pdf Implementation Details (Interpreters) A basic Interpreter object that sub-classes from BaseObject is provided in the Context Toolkit. It allows other components, via their instance of BaseObject, to: queryInAttributes: inspect the context types that an interpreter instance can interpret; queryOutAttributes: inspect the context types that an interpreter instance can output; and, interpret: request interpretation. Fig. 23. Implementation Details (Aggregators) When an aggregator is started, it can either locate appropriate widgets via the Discoverer or it can use a list of widgets provided to it at runtime. The aggregator acts as a mediator between other components and these widgets. When another component wants to communicate with any of these widgets, it can do so via the aggregator. When inspected, the aggregator’s attributes, callbacks and services are a union of these widgets’ attributes, callbacks and services, respectively. At runtime, the entity for which the aggregator is responsible is specified. Implementation Details (Aggregators) For example, this could be “username = Anind Dey” or “location = Atlanta” or “device name = munch.cc.gatech.edu”. This specification is used as a condition when subscribing to relevant widgets, allowing the aggregator to only receive information about its entity. A final difference between aggregators and widgets is in the way that they locally cache context data. A context widget caches the last data that its sensor provided it. An aggregator’s local cache contains an entry for each type of context that the aggregator is receiving, which potentially comes from many callbacks. Rather than overwriting the cache when a new callback is received, only the relevant entries are overwritten. Fig. 25. BUILDING APPLICATIONS WITH THE CONTEXT TOOLKIT In/Out Board and Context-Aware Mailing List: demonstrates use of simple widget that is reusable by multiple applications and demonstrates how applications can evolve to use different sensors; DUMMBO: demonstrates evolution of an application from being non-context-aware to being context-aware; Intercom: demonstrates a complex application that uses a variety of context and components; Conference Assistant: demonstrates a complex application that uses a large variety of context and sensors. context awareness1\dey-thesis.pdf Context Broker Chen:G:\context awareness1\intelligent broker.pdf a context broker has the following responsibilities: provide a centralized model of context that all devices, services, and agents in the space can share, acquire contextual information from sources that are unreachable by the resource-constrained devices, reason about contextual information that cannot be directly acquired from the sensors, detect and resolve inconsistent knowledge stored in the shared context model, and protect privacy by enforcing policies that users have defined to control the sharing and use of their contextual information. Context Broker consists of four modular components. Each component provides distinctive functions for supporting persistent data storage, context reasoning, context acquisition, and privacy protection. Context knowledge base. This component manages the storage of the context broker’s knowledge. This knowledge includes the ontologies for describing various types of contexts, the ontology instance data of the acquired contextual information, and the meta-data for describing the storage structure of the represented knowledge. Context Broker Context-reasoning engine. A logical inference engine for reasoning over the acquired contextual information. The function of this engine includes interpreting context based on the acquired sensing data, aggregating the contextual information from multiple sources using ontologies and domain heuristics, detecting and resolving inconsistent information. Context-acquisition module. This component is a set of library procedures for acquiring contextual information from sensors, agents, and the Web. It hides the lowlevel context sensing implementations from the highlevel functional components. This middle-ware system design is aimed to improve the reusability of the context sensing procedures. Context Broker Privacy-management module. This component manages the users’ privacy policies and controls the sharing of their private information. It is responsible for enforcing the appropriate privacy policies when the context broker attempts to share certain user information. When sharing information, if it is forbidden by some user’s policy, this components also helps to guide the logical reasoning to adjust information granularity. Chen:G:\context awareness1\intelligent broker.pdf Implementation Blueprints It addresses the following engineering issues: the development of ontologies for building a context broker, the service discovery and communication between a context broker and computing entities with heterogenous network interfaces, the deployment and administration of a context broker, the single-point-of-failure scenario in a distributed system. Chen:G:\context awareness1\intelligent broker.pdf Applications of CoBrA Intelligent Personal Agent Projector Tracking Service Visitor Assistant Service Chen:G:\context awareness1\intelligent broker.pdf ONTOLOGIES Chen:G:\context awareness1\intelligent broker.pdf the SOUPA ontology and the CoBrA ontology The CoBrA ontology (COBRA-ONT) is a smart meeting ontology for prototyping CoBrA applications. In particular, the ontology defines the vocabularies for describing the meeting events of the eBiquity Research Group at UMBC. It was first released in March 2003. After the first release, there were five other incremental releases of the revised ontology. The SOUPA ontology, on the other hand, is a standard ontology for supporting pervasive and ubiquitous computing applications. This SOUPA ontology project was initiated by myself and an international group of researchers who are part of the SemanticWeb in UbiComp Special Interest Group (http://pervasive.semanticweb.org). The Web Ontology Language OWL ONTOLOGIES SOUPA Ontology: Fig. 4; CoBrA Ontology Chen:G:\context awareness1\intelligent broker.pdf CONTEXT REASONING CoBrA supports two kinds of context reasoning: reasoning for building a shared model of context, reasoning for maintaining a consistent model of context. The first kind of reasoning involves the use of inference to interpret sensing information, the other kind of reasoning involves the use of inference to detect and resolve inconsistent information. CoBrA’s context reasoning with a rule-based logical inference approach: G:\context awareness1\intelligent broker.pdf PRIVACY PROTECTION G:\context awareness1\intelligent broker.pdf IMPLEMENTATIONS G:\context awareness1\intelligent broker.pdf Functional Architecture of Service Life Cycle Service life cycle: the procedures and steps intended for the design, implementation, activation, operation, and withdrawal of a service. Divided into service creation & service management functional areas Boundary: when the service is to ready to be deployed. Operational phase: activities of deactivation, reactivation, removal, migration, replication, and updates Functional Architecture of Service Life Cycle Service management Install and remove services in distribution points A list of currently installed services must be accessible (required data identifying the code: service name, vendor, version, interface) Decide in which execution environment the code will be service activated. (the service itself may decide on when to carry out a deactivation, suspension, and resumption) Need to maintain a mean of handling configuration data in a location-independent way. Context-Aware Services Life Cycle Model (Fig.3.1) Service Requirement Definition Epoch Service Definition Configuration of necessary System components Incoming Subscription Customization Epoch Customized Service Code Configuration of necessary system components Code Installation Invocation request Invocation Epoch Service Code Execution Operation Epoch Service Code Operation Service Assurance Functional Architecture of Service Life Cycle Service definition: all the functionality fulfilling the requirements + the system configurations necessary to provision the service. Finishes with the dispatch of the service definition and the configuration specifications to the appropriate components of the provisioning system. Customization epoch: Establish an agreement between the new subscriber and the service provider. Establish a subscription, produce & install the customized code in the system components New infrastructure configuration. Functional Architecture of Service Life Cycle Service offered to a single consumer or a group of customers without explicit subscription escape this epoch. Invocation epoch: In response to an incoming invocation request (from user by a signaling protocol, or from the infrastructure by specific alarms) Services may be designed such that no invocation request is required for their use: immediately after customization/definition epoch; based on service schedule detailed within the service definition. Functional Architecture of Service Life Cycle Operational epoch Supported by infrastructure mechanisms that implement the APIs used by the service. Efficient service delivery is ensured by assurance mechanisms that monitor performance and change the configuration of the infrastructure, and of the service itself, as required. The cessation of the execution of code at certain locations at certain points in time and restarting of code at other locations: support mobility without flooding the whole network with pieces of personalized code. Service Life Cycle Decomposition (Fig.3.2) • Active for handling service definition & customization at the corresponding epoches • Configuration & performance management of the created services •Policies are set during Service Definition epoch •Configuration management is active during the service operation epoch and decides the terms of service execution. •Performance management is at the same epoch and ensure correct operation of executing services • Active for executing & supporting service operation (service operation epoch). It supports service operation by providing several APIs. Service creation Policy-based service management configuration performance Service execution Service Layer Functional Architecture (Fig.3.3) Service Creation Functions Service Customization Subscription agreement Code & Policy Generation Engine Set of Policies Code Execution Controller Invocation Service Listener Service Definition Technology-specific Service code Service activation requests Code Distributor Triggering message CAS Authoring Triggers Service Execution Monitoring & config. data Service Assurance Service Management Functions • execution environment provide a platform for service code execution; APIs that offer services access to the capabilities of the infrastructure; communicates with SA to exchange monitoring and configuration data. • the results of monitoring are fed to the Performance Asserter of SA that will decide the enforcement of proactive and reactive measures Service Layer Reference Points (Fig.3.4) CAS Authoring Consumer Request Service Customization Agreed Service Subscription Available Modules Service Definition Document Code and Policies Generation Engine Management Policies Customized service code Service code URL Code Execution Controller Code Distributor Service Activation Request All Pertinent Service Management components Code Installation/ Maintenance Execution Environment Monitoring Data Activation trigger Reactive Configurations Invocation Service Listener Signaling, Alarms Proactive policies Service Assurance Service Creation Service creation process does not need to follow a waterfall model because sequential phases is adequate to the development of such kinds of services. CAS authoring To produce a coherent & complete service definition & to provide a tool to assist the CAS creation administrator to compile the service definition. CAS Authoring Component Functionality (Fig.3.5) Authentication and Authorization Selection of Action Selection of Service /Service Type Modules Selection Complex Attributes Definition Attribute Value Association Service Definition Document Production Service Definition Document Validation CAS Authoring Component Architecture (Fig.3.6) AA Policies Module Documents Skeleton configurations Modules Catalogue GUI Login and Navigation GUI Complex Attrs Templates Service Definition GUI Complex Attributes Definition GUI Validity Guardian Service Definition Document Validation Rules XML-based document Service Customization Component Functionality (Fig.3.7) Connection Establishment Selection of Action Selection of new Service/ Existing customization Service Customization Definition Negotiations Between consumer provider Establishment of Agreement Service Customization Component Architecture (Fig.3.8) Consumer Service Customization GUI Provider’s Negotiation Engine Authentication Authorization Accounting AAA Policies Request Validation Validation Policies Resource Based Admission Logic Multiple instances of CC GUI may exist, but connect to a single PNE Subscription Maintenance PNE should be multithreaded to support multiple simultaneous consumers requests Agreement Deployment Agreed Service Customizations Monitoring date Subscription Repository Code & Policies Generation Engine Component Functionality (Fig.3.9) Receive Service Customizations Generate Customized Code Generate Customization Policies Technology independent Receive Service Definition Document Parse SDD to produce Service Logic Generate Service Logic Code Dispatch Code/ policies Technology independent Parse SDD to produce management policies Generate Service Policies Code & Policies Generation Engine Component Architecture (Fig.3.10) One for each distinct technology Service Logic Parser Service Logic Date structure Code Generator Code Dispatcher technology specific code Service Definition Document Service Customizations Management Policies Parser Management Policies date structure Data structures are implementation technology independent Policies Generator Policies Dispatcher Service Management Starts when the service code is ready to be deployed in a distributed execution environment. The set of functions is heterogeneous: not easy to have closed solutions applicable to any environment. structurized as a service itself. Major requirement: autonomy Need to clarify the dependencies between the highlevel application and its constituent parts and resources. Abstract service models, tools, and languages are used to address this issue. Code Distributor Component Architecture (Fig.3.11) Service Code and Customizations Code Code & Policies Generation Engine Code Installer Translation Policies Code Storage Code Maintenance URL Selector Optimum URL Code Execution Controller Code Distributor Twofold in output: code installation & maintenance, service code URL selection Criteria for selection of storage points: provided by management policies acquired during the service definition epoch, or general policies established by overall system administration decisions. Maintenance function is always active to ensure that the installed code is stored consistently within the infrastructure. Keeping a record of code versions, adding/delete code, relocating code for optimizing its distribution, an updated list of the URLs of the available code Code Distributor URL selection mechanism URL requests from CEC ask for the most convenient URL of a specific customized service given a list of the intended execution points for this service. Selection: based on the installed code, the URL list, and the management policies defined to affect this selection. Code installer: A listener for code installation requests Maintain a catalog of all the available CS points Catalog + management policies generate a list of target storage points for the code in question. Code Distributor CM Notified by the CI for every new installation and keeps an updated list of the code distribution. Removal, update, and redistribution requests are also served. Implementation: Centralized component. A GUI is needed: update available code storage points, monitoring and human-driven management facilities for the installed code. CS could be implemented as a file structure, while the code installation could be achieved using FTP. An LDAP-based storage and retrieval system could be an alternative solution. Code Execution Controller Component Functionality (Fig.3.12) Receive Invocation trigger Deduce Service Deduce Subscription Deduce run time Arguments Authentication Admin Authorization Resource Based Authorization Select Execution Node Retrieve Optimum URL Request Execution Code Execution Controller Component Architecture Invocation trigger messages are uniform Invocation Triggers Listener (Fig.3.13) Invocation resolver AA Policies AA Engine Service, Subscription, Arguments Deduction Policies Proactive Assurance Enforcement Policies Execution Point Selector URL retrieval Activation Request Composer/Dispatcher Activation Request message Distributed execution Policies Code Execution Controller Centralized scheme vs. multiple instances The latter is to optimize performance, increase scalability, and improve resilience of the system The IL must be configured in the latter case with the necessary policies for selecting the appropriate CEC Invocation Service Listener Component Functionality (Fig.3.14) Triggers Captivation Triggers Translation Composition of Activation Trigger Message Selection of appropriate Code Execution Controller Activation Trigger Message Forwarding Invocation Service Listener Component Architecture (Fig.3.15) Connected to a signaling termination server or to an infrastructure capability Triggers Listener Trigger Translator Invocation information is formalized within the message as a set of parameters Invocation Trigger Message Forwarder Activation Trigger Message Translation Policies Code Execution Controller Selection Policies Invocation service listener Each ISI is programmed to recognize only a specific type of trigger (e.g., SIP messages) Translation policies are set by the system administrator and are intended to configure the IS in a flexible and dynamic manner, thus making their functionality more generic. A trigger type may be served by several ISIs to reduce trigger transmission delays, network traffic, and balance server loads. Triggers: Infrastructure alarms can be captured by connecting to appropriate context brokers Specific protocol signals: captured by termination servers Service Assurance QoS perceived by the end-user Service accessibility performance Service retainability performance Service integrity performance SA automatically detects and corrects network and service problems during service life time, in order to comply with SLA, using policies to achieve this in an efficient manner. Refer Fir. 3.4 for SA’s interaction Policy-based features of SA contribute to make CAS assurance a fully automated process, allowing high-levels of flexibility and dynamic configuration of service monitoring and problemsolving actions SA: service monitoring Performance monitoring Threshold data monitoring Utilization, performance, reliability, congestion Mechanism to configure thresholds applied to serviceeffecting measurements, and to report the threshold violations to the interested parties via alarm mechanisms. Fault data monitoring Report the occurrence and clearance of network or service failures. Failures at the network or service levels are reported using notifications and alarms. SA: service control actions Reactive actions (executed once an SLA is violated) vs. preventive actions (performed to prevent SLA violations from occurring) E.g.: manipulation of link traffic overloading Different levels of priority in the actions and therefore different alarm level. Management actions Monitoring state actions: do not explicitly resolve SLA violations, but help to identify and avoid them. changing threshold levels, activating/deactivating alarms,… SA: service control actions Distributed components of SA: Monitoring entity actions: are taken before or after (proactive vs. reactive) the SLA are violated, in 1st case to avoid the SLAs violation, and in 2nd case to restore a normal service performance. activating auxiliary links when bandwidth problems arise. SQM, PM, NAC (each node has its own instance) Centralized components of SA: AM, SLAM, FM (need information from different nodes; may have multiple instances) Service Assurance Component Architecture (Fig.3.16) Network Inventory Repository Network Action Component Assurance Manager SLA Manager SERVICE PERFORMANCE ASSERTER Service Quality Manager Fault Monitor Performance Monitor MONITORING COMPONENT Service Policy Repository Context-aware Services on the Network Layer Network Layer Requirements for Context-Aware Services The paradigm of simple network and sophisticated routing and endpoints cannot support modern services that require QoS support in a scalable costefficient way. Network Context Information: the information that describes the state of the network from several aspects like load, availability, best routing, etc. used by services to allow them to deliver the required service behavior to users and to improve their performance composed of different pieces of information, residing in different network locations. Network Layer Requirements for CAS (cont.) 3 critical aspects in the ability to provide the needed NCI to networked services: The ability to extract pieces of the needed local information (get information from network elements regarding their local state and the state of the network) Create a general network layer view from the local pieces of information. The delivery of the needed information to the different services and the different parts of the service logic, which might be executed in a distributed way. Main obstacles: efficiently collect NCI from the network and process and deliver it to the requesting overlay network as required. Network Layer Requirements for CAS (cont.) by ‘on request’ basis: only need to monitor and collect information that is requested by a client of a CAS. client-dependent Need to reduce overhead traffic as it competes for the same bandwidth with customer traffic Besides acquiring NCI, CAS often needs to take action as to change network configuration in order to be able to deliver the service, or in order to make it more efficient. need for a mechanism to allow the service to take actual actions with respect to the network. (reconfiguration of parameters in one or more NEs, or setting up filters blocking certain packets in different network locations, etc. Done Current State of Service-aware Networks The ability to collect, process, and disseminate NIC, together with the ability to actually change the network behavior by taking actions, is a key ingredient in the ability to design and implement complex CASs in a scalable and efficient way. Service-aware network (SAN): Introducing openness through APIs into networks to increase: flexibility, programmability, protocol adaptation, reusability, scalability, reliability, security and support for services. The adaptation process take place in the middleware monitor events in the underlying network and adapt their internal behavior by either changing specific components’ behavior or dynamically reconfiguring the network. Current State of Service-aware Networks framework of open network: Parlay/OSA: Consists of a set of APIs that enable third parties and network operators to create new applications and services, which have real-time control of network resources. Provides the resource location, authentication, and authorization functions required for external applications to gain access to network-based Parlay services. Parlay services (e.g. generic call control service) offer network capabilities to applications, enabling them to dynamically route and reroute media streams. Current State of Service-aware Networks Network overlay (towards SAN): Applications should not have to deal with the underlying real network infrastructure. A set of nodes (servers, end-user nodes) and virtual links are involved in specific applications. Acts as a specialized middleware. Two key elements: Techniques to efficiently map the overlay abstraction to the underlying resources The management of the overlay, i.e., the mapping policy, configuration and reconfig., performance monitoring, etc. Can be customized or optimized for a single service or a service family, thus creating a variety of overlays. Current State of Service-aware Networks Peer-to-peer overlay networks: CAN, Chord.. Benefit: enable service components deployment in a flexible, scalable, and decentralized way. Key function: a distributed hash table, which allows data to be uniformly diffused over all the participants in the P2P system. Separation between services and network infrastructure results in some loss of interworking between the two layers; the programmable network technology offers some remedy to this inefficiency. Req. for NCI Collection and Dissemination Collect, process, and disseminate NIC at different points of the network: (two aspects) The ability to access local network level information. Process the information in the network level and the disseminate the information to the prospective consumers (the services who need this information) Access to local network level information At different network locations Varies depending on the specific service and the specific network device at hand Req. for NCI Collection and Dissemination Usually accessible either via a standard SNMP agent or using a proprietary CLI. Needs to be abstracted so that upper level applications or entities will not have to worry about dealing with each local devices in the form of an information model and a set of APIs based on such an information model APIs support unified queries regarding network level data efficiency could be provided by a mediator or a set of mediators that have the necessary intelligence to process information taking into account characteristics like the location of information, its volatility, the protocol that understands or is best suited to the specific device, and the grouping of several atomic requests into a combined one to reduce the network overheads. Req. for NCI Collection and Dissemination Gathering and disseminating global network information Provider-consumer approach for the players. Context Information Distribution System (CIDS) binds consumers and producers, and distributes the contexts using well-defined APIs. A system that has endpoints (mediators) at each relevant location to provide the means for efficient information exchange.. The nodes of the CIDS can communicate using a variety of delivery options. enhances the scalability and performance of dissemination Figure 4.1 Context Information Dissemination System(CIDS). Network Node SLO Consumer API Producer API Context Mediator Network Node Network Node SLO Consumer API Producer API Context Mediator Context Dissemination Scheme SLO Consumer API Producer API Context Mediator Req. for NCI Collection and Dissemination Context Information Sources (CISs) Producers providing raw context information from network nodes (local MIB variables), from different mobile access technology adapters (WLAN, GPRS), from other wrappers (weather sensors),… Complex Computation Objects (CCOs) Provide complex information by aggregating elementary context. Deployed (permanently or on-demand) in the EE of the network nodes and registered with the CMs through the Producer APIs Req. for NCI Collection and Dissemination Service Logic Objects (SLOs) Context consumers Instances of the server logic During the operation epoch, each SLO issues requests for the acquisition of context information. Context Mediators (CMs) Act as the third party players between producers and consumers. Provide two types of API: producer APIs and consumer APIs Producer APIs include the interface that enable context producers to publish information they provide. Req. for NCI Collection and Dissemination The heterogeneity of the different context sources is hidden Only local CM needs to be updated with the new interfaces when a modification of the context sources occurs. Consumer APIs enable an SLO to access context information through either pull requests or notification events. Req. for Network Level Control API to reconfigure the network in order to achieve the service goals. Local actions: solely performed at a single network element. Changing a configuration parameter in the local network device, setting up a filter to block certain traffic, changing QoS priorities,... Global actions: performed in more than one network location. Setting up an IP tunnel or changing an OSPF characteristic in an entire region. Req. for Network Level Control To efficiently deliver CASs, allowing code to perform such action in the network layer presents a serious security risk: malicious code, code bugs. Solution: mediator Abstract the action from the service logic Deal with the security and safety issues in a unified way. Handle network-level actions and their atomicity by monitoring the status and rolling back the operation Action Mediator Provide API to the SLOs to realize the necessary functions. can be implemented by programmable network techniques A mediator exists in each relevant network node and perform global action by cooperating among the mediators, where each one is in charge of the actions in the local device. Security Considerations Authorization: CAS get permission to access a CI Authentication: certify CIS’s identity, ensuring that the information source is trustworthy. Encryption Different levels of security: by combining different security mechanisms CAS Creation and Management— System Architecture and Design Consideration Figure 6.1 Overall CONTEXT System Architecture. Consumer Scripts/Users Provider Scripts/Users Service Layer High-Level Service APIs Policy-based Service Management Service Subscription/ Customization Server Service1 Service N Active Application Network Platform Low-Level Medium Specific APIs Context aware Service Creation Server Active Application Layer IP Layer Network Element Abstraction GPRS Layers considered in CONTEXT LAN WLAN Context System Architecture 3 domains: service layer (SL), Active Application Layer (AAL), IP domain SL: the modeling of the context of services and the services themselves, and the required framework for the service creation and management. AAL: with appropriate APIs to control the IP domain (configuration of routers, QoS setting,…) allows the actual delivery and management of CASs: configuring routers to intercept-specific packet types or to alter routing, traffic shaping, and QoS settings. Context System Architecture ALAN (Application Layer Active Networking): Close to the idea of mobile agents Divide functionality into AAL and IP domains lion’s share of network-related services beyond packet delivery can be deployed without affecting the behavior of IP-packet processing in routers. Promote robustness and efficiency (IP layer remains simpler and faster) Figure 6.2 Service Layer Architecture Service Layer Overview Service Layer: Modeling of CASs and context information Creation and management of such CASs Besides components in Fig. 3.3: additional components dictated by policy-based management approach for service management and the DINA active platform for service execution Policy Manager: seeing that The prerequisites of the enforcement of the policies are fulfilled The policies are followed in all locations where they should Only up-to-date policies are present in the system Service Layer Overview Policy Decision-Making component (PDMC) Evaluating policy (accessing the system state changes against the policy rules and initiating suitable reactions) Action Consumer (AC): Undertake specific actions under the instructions of PM (bases its instructions on the decision of PDMC) Code Distributor, Code Execution Controller, Service Assurance The actions may be proactive or reactive Service Execution Components Execution environment of CAS Code storage components : distributed and based on DINA platform (the active nodes of an IP network) For the maintenance of the code base and for the sake of user and code mobility. May be a dedicated server or a DINA node Decision on the storage location belongs to the policies defined for each service. Condition evaluator components (CE) Provide PDMC with the information for evaluating policies. Each time a new policy is received by PM the relevant CEs must be configured/activated Exist in the form of active DINA nodes (nodes’ internal DINA brokers offer the interface to receive information) Figure 6.3 Service Layer Components’ Interfaces. Consumer Request CAS Author Service Customization CAS Authoring Consumer Request Consumer Request Code and Policies Generation Engine CAS Customizations System Capabilities Code Installation/ Maintenance Code Distributor Management Policies Trigger Action Code Execution Controller Code execution message Service Assurance Proactive Policies Code Storage Reactive/Proactive Configurations Code SLO DINA Node Broker I/F Policy Manager Actions Decision Policies Policy Decision Making Configuration Assurance Monitor Notification CE Service Layer Implementation Considerations CAS Creation, Service Customization, and Code & Policies Generation Engine are implemented as stand-alone applications. Modeling of CAS: XML and XML schema GUIs: Java Repositories containing CAS info: database Information flow between components: SOAP Code generation: Java to produce DINAcompatible code Produced code: placed into an Apache server to make it available to the code distributor Why Policies Policy: an administrator-specific directive that manages and provides guidelines for how the different network and service elements should behave when certain conditions are met Benefits: improve scalability & flexibility, simplicity (centralized configuration, simplified abstraction) Policies are predefined in XML Predefined policies as templates, personalized by the user’s preferences and context Objectives of PBMS Automate management and do it as high level as possible Tailored to different users Encompasses the expression and sequent creation of portions of the logic of CAS Manages various aspects of the created CAS (deployment, invocation, execution, maintenance, assurance of services) Expressed in XML, and personalized by user subscriptions PBMS Characteristics of PBMS Open Flexible Modular Scalable Distributed Main functional blocks: code distribution & maintenance, code execution, service invocation, service assurance Technology & policy specific components: Action Consumers and Condition Evaluators CEs may be a set of different components specialized for particular tasks installed in different points of network. (e.g., monitoring different performance or quality parameters, different kinds of invocation signals,..) Figure 6.4 First Approximation of the Generic Policy-Based Management System to the CONTEXT Service Lauer Functional Requirements. Figure 6.5 IST-CONTEXT Policy-Based Service Management Layer Architecture Domain-specific Policies High-level description of policy: IF (cond. 1) [AND/ OR (cond. N)] THEN (action1) [AND (action n)] Just control the service life cycle, never the logic of the service All actions and conditions must be implemented in different components playing the role of ACs and CEs 4 functional domains: service code distribution, maintenance, invocation & execution, assurance Domain-specific Policies service code distribution : Will be processed when new customized service code arrives to distribution component. Drive the deployment process of needed service. Specify the specific configuration, required resources, and criteria for optimum storage point selection for the service. service code maintenance: Will be enforced when maintenance event arise: relate to new service version, service expiration, storage points resources under desirable margins, high load of invocation petitions, etc. The action enforced will be service code removal, service code update, service code redistribution. Domain-specific Policies service code invocation & execution: monitoring of variables or events that start the execution of a CAS. I1 abstraction level: raw info. coming from the system services to SICE (userId, password, raw data measurement) I2: info. coming from SICE to DM (filtering or mapping I1 level information) I3: the info. identifies (a) which customized service(s) to execute, (b) the initial conditions (run-time arguments), and (c) where to execute the customized service code determined for execution. (corresponds to the input parameters required by the action undertakers (ACs) Service assurance: SA initialization policies SA execution policies SA finalization policies Functional overview of SA Main principles SA behavior for a certain CAS is controlled directly by the CAS author through assurance policies efficiency SA design avoids disturbing the CAS with assurance issues, making assurance as transparent to the CAS as possible. flexibility, and allowing configuration at both deployment time and run time. Functional decomposition Service monitoring: Service management Figure6.6 CONTEXT Policy-Based Service Management Layer Architecture. SLO AS AC DMC SOAP to DMC ASCE ServiceMonitMeasure Service Monit mng xsit Srv xslt Sup xsit XML to console SLO specific logic XML monitored data Local DINA info Figure6.7 Flow Diagram. SLO startMonitoring ServiceMonitMeasure AS AC new monitor set-up DMC new Assurance Message ASCE SOAP to DMC stopMonitoring ServiceMonit setSpecific mng xslt srvx slt getAssurance Data new monitor set-up sup xslt throwAlarm XML to console SLO specific logic XML monitored data Local DINA info