Emergent Semantics
and
Experiential Computing
Ramesh Jain
CTO, PRAJA inc.
and
Professor Emeritus, UCSD
rjain@praja.com
© Ramesh Jain
Semantics

Meaning
 Negotiation process: requires agreement
among all involved participants
 Every User has their ‘personal ontology’
 Personal ontology can not be matched
with the system ontology in one step (one
query) --- requires an emergent process.
© Ramesh Jain
Information Sources
Video
Audio
Data and
Statistics
Text
Sensors
Index
Index
Index
Index
Index
Feature Indexed
Early Developments
XML
Powerful Data
Models: Relational
© Ramesh Jain
The Semantic Gulf

Data is organized based on data
characteristics.


Efficiency and Scalability are primary
User intentions and context are not
considered.

Query environment does not maintain the state
of the user.
© Ramesh Jain
Access: Impedance Mismatch


Computers are millions of times faster than
humans in arithmetic and logic.
People are millions of times better than
computers in perceptual and conceptual tasks.



Any one year old can recognize objects!
Current computing environments were designed
for people to serve computers.
Consider computers and humans part of a
symbiotic system.
© Ramesh Jain
Semantic Indexing
Video
Audio
Sensors
Data and
Statistics
Text
Index
Index
Index
Index
Index
Index and Link based on Event-graph
© Ramesh Jain
Information Assimilation
.
Semantic procedural
updating
ModelBased
Updating
And
Linking
EventBase
Semantic links used to
display unstructured data
Navigation
and
Visualization
Environment
User
© Ramesh Jain
Features of Experiential Environment
Natural Action Responses – No unusual
metaphors
 Query and Presentation spaces must be
the same (What-You-See-Is-What-You-Get)
 Continuity of User State and Context –
Minimize Latency; Feedback
 Multimedia Immersion and Exploration

Video games should be the model.
© Ramesh Jain
Example Applications
© Ramesh Jain
DocumentWeb: Information Age
More than Five Century old
Legacy of Gutenberg
Continues…
Despite all advances in technology…
© Ramesh Jain
Strategic Inflection Points
Immersive
Experience
Events on Web
Contextual
Search
(Experience)
Ubiquitous
Devices
Documents on Web
Semantic
Search
(Information)
Updates
and alerts
Keyword
Search
1995
2000
2005
2010
© Ramesh Jain
EventWeb: Experience Age
Family
Sports
Office
Fun
Finance
Knowledge
Personal
© Ramesh Jain
Top 5 Misconceptions





All users should have Ph.D. in Ontology.
User queries are Context-Free.
Users must be allowed only one query.
Image and Video semantics is in features of
FULL images or video.
All information is alpha-numeric.
© Ramesh Jain
Thanks.
© Ramesh Jain
Data Information Experience
© Ramesh Jain
Information Integration: 1
Query
Parsing
And
Report
Generation
User
© Ramesh Jain
© Ramesh Jain
Time Machine - Replay
Example©Apps
Ramesh Jain
Information System Evolution:
Databases
Queries
Query
Translation
And
Response
Declarative and Stateless
Users
© Ramesh Jain
Information System Evolution :
Personalization
Personalization
Filters
Queries
P- Queries
Query
Translation
And
Response
Declarative and Stateless
Users
© Ramesh Jain
Information System Evolution :
Contextual Systems
Personalization
Filters
Context
Queries
P- Queries
Query
Translation
And
Response
Declarative and Stateless
Users
User
States
© Ramesh Jain
Contextual Navigation
Query
Transformation
Query
Context
Data
Presentation
System
What-You-See-Is-What-You-Get (WYSIWYG) Search.
© Ramesh Jain
Kalman Filtering
Data
Processing
Model
© Ramesh Jain
Basic Concept

Model: Mathematical State and other relations

Current State = F (Previous State, New
Observations)

Models states and transitions based on each
source of data
© Ramesh Jain
Symbolic Kalman Filtering
The Model is Hybrid – combination of
symbolic and mathematical.
 Each Data Source is an independent
observation source.
 Model is updated based on its current
state and new observations.

Essential for live and sensory data assimilation.
© Ramesh Jain
Entities and Events
Entity
Name
Event
Name
Duration
Location
Attributes
Processes
(Services)
Objects and Entities
are static.
Attributes
Data-streams
Processes
Adjacent States
Related Links
Events are dynamic.
© Ramesh Jain
Modeling a Domain

Event Graphs: Capture relationships
among events and entities.
 Event Transitions: Conditions and
probabilities of transitions.
© Ramesh Jain
Experience:
Direct Observation of
or Participation in
Events
as a basis of knowledge
© Ramesh Jain
Objectives

Experiential environments are the next
major technology inflexion point.
 Unified indexing of assimilated data is
essential to implement experience-centric,
rather than current information-centric,
systems.
 Semantics is ubiquitous and emerges with
symbiotic interactions among a user and
the system.
© Ramesh Jain
Information Integration: 2
Integrated
Database
User
© Ramesh Jain