Spatiotemporal and Thematic Semantic
Analytics
Matthew Perry
PhD. Student
Kno.e.sis Center, Wright State University
Knowledge Enabled Information and Services Science
Three Dimensions of Information
Thematic Dimension: What
Temporal Dimension: When
North Korea detonates nuclear device on October 9, 2006
near Kilchu, North Korea
Spatial Dimension: Where
Knowledge Enabled Information and Services Science
Using named relationships to connect thematic entities with spatial
locations in a variety of meaningful ways (different contexts)
E2:Soldier
E4:Address
located_at
lives_at
located_at
lives_at
E6:Address
E1:Soldier
assigned_to
Georeferenced Coordinate
Space
(Spatial Regions)
occurred_at
E7:Battle
participates_in
E8:Military_Unit
E8:Military_Unit
participates_in
E5:Battle
Named Places
assigned_to
occurred_at
Residency
Battle Participation
KnowledgeDynamic
Enabled Entities
Information and Services Science
Spatial Occurrents
E3:Soldier
Temporal Properties of Paths
Soldier_1
Soldier_2
assigned_to:[3, 12]
assigned_to:[6, 13]
Platoon_45
assigned_to:[14, 20]
assigned_to:[19, 25]
Soldier_4
Soldier_3
Which soldiers were members of Platoon_45 during the interval [5, 15] ?
Which soldiers were members of Platoon_45 at the same time ?
Knowledge Enabled Information and Services Science
Example: Bioterrorism
assigned_to
E10:Docto
r
E9:Base
Spotted Close in Time
E1:Soldie
r
stationed_at
member_of
[0, 10]
E7:Platoo
n
causes
participated_in
[4, 6]
spotted_at
[3, 5]
E11:Locatio
n
E14:Battl
e
Near in Space
E5:Terroris
t
member_of
sign_of
After the Battle
carried_out
[0, 2]
E4:Chemica
l
E3:Diseas
e
exhibits
[8, 10]
E2:Sympto
m
E8:Soldie
r
E6:Attack
used_in
[0, 2]
E13:Soldie
r
E12:Platoo
n
participated_in
Knowledge Enabled Information and Services Science
member_of
Background
• New types of applications exploiting named relationships
between entities (semantic graphs)
– Data Mining – Link Mining, Graph Mining
– Semantic Web – Semantic Analytics
• Analysis of relationships in Large RDF graphs
• Detecting Conflict of Interest, Collaboration, Insider Threat
Problem
Knowledge Enabled Information and Services Science
Semantic Connectivity
•
Two entities e1 and en are semantically connected
if there exists a sequence e1, P1, e2, P2, e3, … en-1,
Pn-1, en in an RDF graph where ei, 1  i  n, are
entities and Pj, 1  j < n, are properties
“Matt”
Semantically Connected
&r1
&r6
“Perry”
&r5
name
“Knoesis Center”
Knowledge Enabled Information and Services Science
name
“Wright State
University”
Semantic Similarity
•
Two entities e1 and f1 are semantically similar if there exist two semantic
paths e1, P1, e2, P2, e3, … en-1, Pn-1, en and f1, Q1, f2, Q2, f3, …, fn-1, Qn-1,
fn semantically connecting e1 with en and f1 with fn, respectively, and that
for every pair of properties Pi and Qi, 1  i < n, either of the following
conditions holds: Pi = Qi or Pi  Qi or Qi  Pi ( means rdf:subPropertyOf
)
We say that the two paths originating at e1 and f1, respectively, are
semantically similar and thus so are the entities e1 and f1
Passenger
Ticket
Corporate Account
“Bill”
&r1
“Jones”
“Fred”
&r7
“Smith”
purchased
&r2
paidby
&r3
Semantically Similar
•
purchased
&r8
paidby
&r9
lname
Knowledge Enabled Information and Services Science
Spatial and Temporal Semantic Analytics
Soldier_6
Soldier_7
Member_of
Spatial Relationship
Temporal Relationship
Axis
Soldier_1
Employed_unit
Position_3
Held_position: [4, 12]
Soldier_2
Member_of
9th SS Panzer
Position_1
Held_position: [2, 10]
Trained_at: [8, 10]
Soldier_3
Specializes_in
101st Airborne
Explosives
Allies
Employed_unit
Position_2
Trained_at: [2, 6]
Held_position: [3, 7]
Camp Claiborne
Employed_unit
Soldier_4
Soldier_5
82nd Airborne
Member_of
Thematic Relationship
Knowledge Enabled Information and Services Science
Trained_at: [1, 5]
Spatial Relationships Between Entities
Examples:
– Which Military Units have spatial extents which are
within 20 miles of (48.45° N, 44.30° E) in the context
of Battle participation?
– Which infantry unit’s operational area overlaps the
operational area of the 3rd Armored Division?
Quantitative Relationships
Qualitative Relationships
Knowledge Enabled Information and Services Science
Temporal Relationships Between Entities
Examples:
– Which Speeches by President Roosevelt were given
within one day of a major battle?
– Who were members of the 101st Airborne during
November 1944?
Quantitative Relationships
Qualitative Relationships
Knowledge Enabled Information and Services Science
Current Work
• Define a Domain-independent Ontology which integrates
Spatial and Thematic Knowledge
– Allows exploiting the flexibility and extensibility of Semantic
Web data models
– Can deal with incompleteness of information on the web
• Incorporate temporal metadata into this model
• Identify and formalize basic spatial and temporal
relationship-based query operators which complement
current thematic operators of SemDis
Matthew Perry, Farshad Hakimpour, Amit Sheth. "Analyzing Theme, Space and Time: An Ontology-based
Approach", Fourteenth International Symposium on Advances in Geographic Information Systems (ACMGIS '06), Arlington, VA, November 10 - 11, 2006
Knowledge Enabled Information and Services Science
Upper-level Ontology modeling Theme and Space
Continuant
Occurrent
Spatial_Occurrent
Named_Place
Dynamic_Entity
located_at
occurred_at
Spatial_Region
rdfs:subClassOf
property
Final Classification of Domain
Classes depends upon the intended
application
Occurrent: Events – happen and then don’t exist
occurred_at:Those
Links
Spatial_Occurents
to theirbehavior
geographic
locations
Named_Place:
Spatial_Region:
Records
entities
exact
with
spatial
static
location
spatial
(geometry
objects,
building)
Continuant:
Concrete
and
Abstract
Entities
– persist
over (e.g.
time
Spatial_Occurrent:
Events
with
concrete
spatial
locations
(e.g.
a speech)
located_at:
Links
Named_Places
to
their
geographic
locations
Dynamic_Entity: Those entities with dynamiccoordinate
spatial behavior
system
(e.g.
info)person)
Knowledge Enabled Information and Services Science
Upper-level Ontology
Continuant
Occurrent
Named_Place
Dynamic_Entity
located_at
occurred_at
Spatial_Occurrent
Spatial_Region
City
Person
trains_at
Speech
gives
Politician
Military_Unit
Soldier
participates_in
Military_Event
assigned_to
on_crew_of
used_in
Bombing
Battle
Vehicle
Domain Ontology
Knowledge Enabled Information and Services Science
rdfs:subClassOf used
for integration
rdfs:subClassOf
relationship type
Thematic Context
Specifies a type of connection between resources in the thematic dimension of our ontology
Schema
on_crew_of
Person
‘John
Smith’
Military_Vehicl
e
on_crew_of
used_in
Bombing
used_in
‘B24#123’
Path Template
‘Bombing#456
’
Person.on_crew_of.Military_Vehicle.used_in.Bombing
‘John Smith’.on_crew_of.Military_Vehicle.used_in.Bombing
Knowledge Enabled Information and Services Science
Thematic Query (ρ-theme)
ρ-theme (G, tc)  {pt}
Example: find all Bombing events connected to ‘John Smith’
through a vehicle participation context
ρ-theme (G, ‘John Smith’.on_crew_of.Military_Vehicle.
used_in.Bombing)
Result
‘John Smith’.on_crew_of.‘B-24#123’.used_in.‘Bombing#456’
‘John Smith’.on_crew_of.‘B-24#123’.used_in.‘Bombing#789’
G = temporal RDF Graph, tc = thematic context, pt = thematic context instance
Knowledge Enabled Information and Services Science
Thematic Contexts Linking Non-Spatial Entities to Spatial Entities
E2:Soldier
E4:Address
located_at
lives_at
located_at
lives_at
E6:Address
E1:Soldier
assigned_to
Georeferenced Coordinate
Space
(Spatial Regions)
occurred_at
E7:Battle
participates_in
E8:Military_Unit
E8:Military_Unit
participates_in
E5:Battle
Named Places
assigned_to
occurred_at
Residency
Battle Participation
KnowledgeDynamic
Enabled Entities
Information and Services Science
Spatial Occurrents
E3:Soldier
Incorporation of Temporal Information into RDF
•
•
•
•
Use Temporal RDF Graphs defined by Gutiérrez, et al1
Models Absolute Time
Considers time as a discrete, linearly-ordered domain
Associate time intervals with statements which represent
the valid-time of the statement
– Essentially a quad instead of a triple
1. Claudio Gutiérrez, Carlos A. Hurtado, Alejandro A. Vaisman: Temporal RDF. ESWC 2005: 93-107
Knowledge Enabled Information and Services Science
Example Temporal Graph: Platoon Membership
assigned_to [5, 15]
E4:Soldier
assigned_to [1, 10]
E1:Soldier
E2:Platoon
assigned_to [11, 20]
E3:Platoon
assigned_to [5, 15]
Knowledge Enabled Information and Services Science
E5:Soldier
Querying in the STT dimensions
• Provide a means to query about spatial, thematic, and
temporal properties/relationships of all entities
• Path Query in the thematic dimension
– Thematic Context
• Associate spatial region with a path
• Associate temporal interval with a path
• Query operators based on properties of and relationships
between associated spatial regions and temporal
intervals
Knowledge Enabled Information and Services Science
Deriving Spatial and Temporal Properties of Entities
ρ-spatial_extent (G, {pt})  {pt, sr}
Retrieves the Spatial_Region connected (through occurred_at or located_at)
to the terminating Spatial Entity of the context instances
Example: Where were the battles in which the ‘101st Airborne
Division’ fought?
ρ-spatial_extent (G, ρ-theme (G, ‘101st Airborne Division’.
.participates_in.Battle))
Result
‘101st Airborne Division’.participates_in.
‘Operation Market Garden’, ‘Geom#123’
G = temporal RDF Graph, pt = thematic context instance, sr = spatial region
Knowledge Enabled Information and Services Science
Temporal Properties of Context Instances
Intersection
[6, 12]
assigned_to:[3, 12]
Soldier#123
assigned_to:[6, 20]
Platoon#456
Range
[3, 20]
Knowledge Enabled Information and Services Science
Soldier#789
Deriving Spatial and Temporal Properties of Entities
ρ-temporal_intersect ({pt})  {pt, [t1, t2]}
Retrieves the interval during which the entire path is valid
Example: Which Soldiers were members of the ‘1st Armored
Division’ at the same time?
ρ-temporal_intersect (ρ-theme (G, Soldier.assigned_to.‘1st
Armored Division’.assigned_to.Soldier))
Result
‘Fred Smith’.assigned_to.’1st Armored Division’.assigned_to.
‘Bill Jones’, [1941:04:15, 1943:02:30]
pt = thematic context instance, [t1, t2] = temporal interval
Knowledge Enabled Information and Services Science
Querying Relationships Between Entities
(Thematic Context Instance tp, Temporal Interval [ti, tj], Spatial Region sr)
3 Spatial Relationship Operators
ρ-spatial_locate
ρ-spatial_eval
ρ-spatial_find
3 Temporal Relationship Operators
ρ-temporal_restrict
ρ-temporal_eval
ρ-temporal_find
Identify 6 major Spatiotemporal Relationship Queries
which can be answered by combining previously
defined operators
Knowledge Enabled Information and Services Science
Spatiotemporal Relationship Queries
Example: When did the 101st Airborne Division come within
10 miles of the 1st Armored Division in the context of
Battle participation
S1  ρ-spatial_extent (G, ρ-theme (G, ‘101st Airborne Division’.
participates_in.Battle))
S2  ρ-spatial_extent (G, ρ-theme (G, ‘1st Armored Division’.
participates_in.Battle))
ANS  ρ-temporal_intersect (ρ-spatial_eval (S1, S2,
distance (S1, S2)  10 miles)
Knowledge Enabled Information and Services Science
Spatiotemporal Semantic Associations
• Define setting as a region of space in combination with an
interval of time
• How is entity X related to Spatial setting S? ( ρ (entity, setting))
Group 1
Account_1234
Fred
125 Broad Street
Jim
Attack Site
How is Group 1 connected to the setting of the expected attack?
Knowledge Enabled Information and Services Science
Spatiotemporal Semantic Associations
How are entity X and entity Y related w.r.t Spatial setting S?
ρ (entity, entity, setting)
Group 1
Group 2
Account_1234
Fred
Jim
125 Broad Street
How are Group 1 and Group 2 connected with respect to the location of the crime?
Knowledge Enabled Information and Services Science
Spatiotemporal Semantic Associations
• Idea of Virtual Links between entities based on
Spatiotemporal information
• Possible definition of rules to define a virtual link type
– Collaboration: entity X and Y are in close ST proximity more
often than a given threshold
– Knows: entity X and Y are in close ST proximity regularly
Knowledge Enabled Information and Services Science
Other Aspects
• How do temporal relationships affect association
semantics
– 2 works_for relationships (overlapping times, disjoint times, etc)
• Complex queries based on all 3 dimensions
– Which location is the most likely storage facility for exfiltrated
weapon material
• Thematic (correct capabilities, linked to correct people)
• Spatial (where was the material last seen)
• Temporal (how long can the material stay out of storage)
Knowledge Enabled Information and Services Science