Managing Context Information in
Mobile Devices
2011. 3. 28.
Panu Korpipää, Jani Mäntyjärvi,Juha Kela, Heikki
Keränen, and Esko-Juhani Malm
VTT Technical Research Centre of Finland
0
Outline
• A context management framework
– Information sources
– Ontology for sensor-based context information
– Inference mechanism concept
• Bayesian reasoning for higher level contexts
• Application programming interface
– Adding context
– Requests and responses
– Subscriptions and indications
– Example application
• Discussion
1
A context management framework
• Motivation
– Mobile device users’ desire
• Access and manipulation of the users’ information
• Services specific to their location, time, and environment
– Gathering context information from various sources
• Sensors, networks, device status, user profiles, and other sources
– Enhancing mobile applications’ usability
• Letting the applications adapt to the users’ conditions
• Problems to achieve true context awareness
– Uncertain, rapidly changing, partially true data from multiple
heterogeneous sources
– Extracting relevant context information by fusing data from several
sensors
– Many factors such as noise, faulty connections, drift, miscalibration,
wear and tear, humidity to degrade data acquisition
2
A context management framework
• Problems to achieve true context awareness
– Overlapping extracted contexts, changing contexts with time, only
partially reliable approximations
• Mobile devices’ dynamic environment
– Learning from multidimensional data
• Impossible easy generalization beyond training data
– Occurrence of problems using even sufficiently reliable contexts
• Users with different ideas of “context”
• Proposed approach to address the challenges
– A uniform mobile terminal software framework
• Providing systematic methods for acquiring and processing useful
context information from a user’s surroundings
• Giving the context to applications
3
A context management framework
• Framework recognizing semantic contexts in real time
– Dealing with uncertain, noisy, and rapidly changing information
– Delivering contexts for the terminal applications in an event-based
manner
– Application programming interface (API) for using semantic context
information
– Using an expandable context ontology to define contexts
• Framework properties
– Blackboard-based approach underlying communication paradigm
between framework entities
– Focusing on mobile terminal capabilities rather than infrastructure
– Framework for the Symbian platform to achieve true device mobility,
high performance, and a broad user base
4
A context management framework
• Four main functional entities of the context framework
– Context manager
– Resource server
– Context recognition service
– Application
• Context manager
– A central server for providing services to other entities
– Running on the mobile device itself
• Black board-based context manager
– Central node storing context information from any source
– Serving context to clients in three ways
• Direct querying the manager to gain context data
• Subscription to various context change notification services
• Using higher-level (composite) contexts transparently
5
A context management framework
• Resource server
– Connecting to any context data source
– Posting context information to the context manager’s blackboard
– Delivering context to the clients according to their subscriptions
• Delivered data’s abstraction level → high enough
• Delivered data’s frequency → low enough (not overwhelming)
– Performing low-level recognition before delivering the data
• Low-level context recognition process flow in the resource server
• Measurement phase
– Reading the sensors and output raw data
• The preprocessing phase builds measurement data arrays that
contain a certain number of samples (quantization of time dimension)
6
and calculates generic features for each time interval.
A context management framework
• Resource server
– Performing low-level recognition before delivering the data
• Preprocessing phase
– Building measurement data arrays that contain a certain number of
samples (quantization of time dimension)
– Calculation of generic features for each time interval
• Feature extraction
– Calculation of more specific context features
• Quantization phase
– Binding the feature values to the real-world context which has a
meaning for a person or an application according to a predefined
ontology
– Using fuzzy sets or crisp limits for quantizing extracted features
– Outputs context atoms (base context units) that applications can
use or recognition services can refine
7
A context management framework
• Resource server
– Two methods for quantization
• Set crisp limits in a true-false labeling with fuzzy membership functions
denoted as μ(x)
• Quantizing environment sound intensity
– Three quantities (Silent, Moderate, and Loud) corresponding to the
three membership functions
• If one of these is true, the others are false.
8
A context management framework
• Resource server
– Two methods for quantization
• Apply a fuzzy set for features, resulting in continuous valued fuzzy
labeling
• μL(x) = 0.7 / Silent + 0.3 / Moderate + 0 / Loud
9
A context management framework
• Context recognition services
– Sharing recognized higher-level contexts
– Addition and removal of recognition services by developers or
processes
– Using a set at a certain time instant or a time series of context atoms
as input
– Returning single higher-level contexts for the context manager
• Summary of framework
– Resource server and recognition service
• Converting an unstructured raw measurement data flow into a
representation defined in the context ontology (human-interpretable
context information) for the applications in an event-based manner
– Framework providing a semantic interface that enables more
systematic and rapid application development
– Efficient reuse of context information compared to using raw
measurement data
10
A context management framework
• Blocks for change detection and security
– Not focusing on the functionalities
– Any change detection services to detect context change
– Security module for checking the trustworthiness of incoming contexts
11
Information sources
• Acquiring context information from many possible sources
– Using a microphone, three accelerometers, two channels for light,
and sensors for temperature, humidity, and touch
– Sensor data provided the starting point
• Device processes that represent mobile devices’ internal information,
such as applications currently running
• The Internet and other networked resources
• Global Positioning System, among the most important sources of
location information
• Internal device processes, which provide explicit information on device
application use and users’ scheduled tasks, preferences, and social
networks
• Time information, used to associate certain events with others and
form event sequences that might relate to a current higher-level
context or predict a future event
– Providing continuous and ad hoc connectivity to local and remote
data
• Supporting various wireless networks, such as GSM and GPRS, and
12
short-range ad hoc networks, such as Bluetooth
Ontology for sensor-based context information
• A common structure for representing information
– Managing the context information systematically
– Design of an ontology for representing sensor-based context
information
• A schema for ontology
– A client-usable, extendable vocabulary that presents the terms for
describing context information
– Using the Resource Description Framework (www.w3c.org) as the
description syntax
13
Ontology for sensor-based context information
• Description of each context using six properties
– Containing at least Context type and Context value
– Context type
• Referring to the context category
• Primary parameter of all subscriptions and queries
– Context value
• Referring to the semantic or absolute “value” of context type
• Usually being used together with context type
• Containing an absolute numerical value or feature describing context
in some cases
– Confidence
• An optional property which describes the context’s uncertainty
• A probability or a fuzzy membership of context depending on the
source
– Source
• Description of the context’s (semantic) source
14
Ontology for sensor-based context information
• Description of each context using six properties
– Timestamp
• Denoting the latest time the context occurred
– Attributes
• Specifying the context expression freely
• Containing any additional details not included in the other properties
• Higher-level context and context atoms links to a recognition service
– Containing the Input context type set for recognizing the higher-level
context
• Three properties for describing recognition services:
– Context type
• Using the category of the higher-level context as a normal context
atom to request the recognition for the type
15
Ontology for sensor-based context information
• Three properties for describing recognition services:
– Context recognition service
• Receiving input defined in Input context type set
• Performing the classification, and returning the higher-level context
– Input context type set
• Subscription of the input context type set to be notified of the changes
• Inferring the higher level context upon change in input type set
• Seven context values in higher-level contexts from a set of 47 audiobased context atoms
16
Ontology for sensor-based context information
• Management of uncertainty of sensor-based information
– Confidence property
• Description of context uncertainty: a context derived from multiple
sensory input only holds true with a certain probability
– Partly true context
• Boundary between two contexts is not clear and discrete
17
Inference mechanism concept
• The conceptual blackboard-based system
– Use of any inference framework for reasoning new contexts or actions
• Context uncertainty described in the confidence property
– Useful in such inference frameworks as probabilistic networks,
clustering, or case based reasoning
• Recognition services
– Input : information from the common data space
– Output : higher-level contexts from either a set of contexts at a
certain moment or a context history
18
Bayesian reasoning for higher level contexts
• Using a naïve Bayes classifier to recognize higher-level contexts
• Naïve Bayes classifier for online context inference and sensor fusion
– Robust method with missing, uncertain, and incomplete information
• Possible to use context data described by the ontology, a vector of
context atom confidence values
• Treating fuzzy membership values as virtual evidence
– Computationally efficient
• Training and inference both have a linear complexity in input data size
– No requirement of background information for modeling
• Exception for choosing the relevant network inputs
• Examples of contexts to be recognized by a naive Bayes classifier
19
Bayesian reasoning for higher level contexts
20
Bayesian reasoning for higher level contexts
• Higher-level context Outdoors from the context atoms
• Indoor/Outdoor Bayesian network
– Classification of the current situation’s context atom confidence
values (vector)
– Conditional probability indicating each input’s probability given the
output
– Learning conditional probabilities from the training data
• Using naïve Bayesian networks to classify 9 contexts of a mobile
device user in their normal daily activities
– Nine contexts measured from a continuous real-life scenario
containing different activities such as driving a car, running, walking,
using an elevator, listening to different kinds of music, and speaking
– The measurement system hardware
• A small sensor box attached to the shoulder strap of a backpack
containing a laptop
21
Bayesian reasoning for higher level contexts
• The classification results
– Indication that naïve Bayes classifier can extract situations fairly well
– Valid only in a restricted scenario
• Experiments under controlled scenarios
– Nine different contexts, and measured by four people
– Car, Elevator, and Tapwater with nearly 100 percent accuracy
– Supervised learning approach
• which defines the target classes and their inputs before training
22
Application programming interface
• Context manager API
– Implemented on the Symbian platform
23
Adding context
• Processing and collecting the context properties by resource servers
• Method for adding context:
ContextAdd( ContextObject )
24
Requests and responses
• Request of context information from the context manager database
– Exposition of context ontology to application developer
– Request context information by the context ontology tree (sub branch)
– Response containing all contexts under the sub branch
– Denoting responses to requests with an arrow (→)
ContextRequest( Environment:Light:Intensity )
→Bright
ContextRequest( Environment:Light )
→{Bright, Natural, NotAvailable}
ContextSetRequest( {Environment:Humidity,
Device:Activity:Position}
→{Dry, AntennaUp}
ContextSetRecognitionRequest(
Environment:Sound:Type )
→Car
25
Subscriptions and indications
• Giving applications the required context information in an event-based
manner
– Subscription to context change notifications
• Examples illustrate how applications use this service
ContextChangeSubscribe(Device:Activity:Placement)
→AtHand
ContextChangeSubscribe(
Environment:Temperature:Absolute )
→21
ContextStartSubscribe( Location:Facility,
MovieTheatre)
→MovieTheatre
ContextChangeSubscribe(
Environment:Sound:Type )
→Elevator
26
Example application
• Using the context ontology to demonstrate the use of sensor-based
context information in mobile terminal applications
– Such as browsing and information presentation in the user interface
– Environment:Light:Intensity, Environment:Sound:Intensity, and
User:Activity:PeriodicMovement
• Environment sound intensity
– Control of the volume of operating tones
• User activity and ambient light level
– Adaptation of font size, screen brightness, and service content
• Context-based information representations of a bus timetable service
27
Example application
• Using fuzzy rules to help guide the application adaptation
• Fuzzy rule definition for adapting font size
– Using the logical operation AND
• Intersection between two fuzzy sets
• Environment:Light:Intensity and User:Activity:PeriodicMovement:
μFONTSIZE (x) =
min(μUSER:ACTIVITY:PERIODICMOVEMENT(x),
μENVIRONMENT:LIGHT:INTENSITY (x)).
– Using the logical operations OR and NOT
28
Discussion
• Framework and context ontology
– Providing a semantic application programming interface for handling
imprecise information from multiple sources
– Many remaining obstacles to the rapid use of many contexts derived
from sensor data
– Need for more sensors and a large data collection to gain enough
information
• Several advantages of high-level context
– Event based communication of high-abstraction-level data
– Communicating low-level data up to a client continuously
• Consumption of more processing power and the limited battery
resources.
– Optimal use of the context manager storage
29