Data Integration on the
Semantic Sensor Web
Alasdair J G Gray
Information Management Group
University of Manchester
Seminar at Imperial College London
7 April 2011
Overview of the Talk
• Motivation: Estuarine Flooding
• Semantic Sensor Web – SemSorWeb
– Requirements
– Architecture
• Ontology-based query access over
sensor, streaming, and stored sources
• Demo
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
1
The Solent
• Strait separating the Isle of
Wight from English mainland
• Complex tidal and wave
patterns
Image: http://www.combinedops.com/
Image: http://www.chandlery-online.com/
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
2
The Solent
Images: http://www.wikipedia.com/
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
3
Estuarine Flood Warning
• Financial implications
– Damage
– Loss of business
• Personal factors
– Emotional
• Flood prediction
– Early warning/forecasting
– Locations
– Severity
• Requires correlating
– Sea-state data
– Weather forecasts
– Details of sea defences
• Response Planning
– Evacuation routes
– Personnel deployment
– …
• Requires more data
Image: http://www.metro.co.uk/
7 April 2011
– Traffic reports
– Shipping
– …
Data integration on the SemSorWeb – Alasdair J G Gray
4
Flood Risk Alert: English
South Coast
• Detect conditions likely to
cause a flood
“Provide details of overtopping
events in the Solent region with
high wind speed observations”
• Need to correlate data from
multiple sources
• Sensor data
• Web feeds
• Databases and other
sources
• Present data model in terms of
the user domain: e.g. Flood
risk assessment
7 April 2011
Real-time
sensor
data
Wave,
Wind,
Tide
Meteorological
forecasts
Flood defences
data (database)
...
...
...
Other sources
Data integration on the SemSorWeb – Alasdair J G Gray
5
Sensor Web Requirements
“Provide flood risk details of overtopping
events in the Solent region with high wind
speed observations”
1. Accurate characterisation of conditions
that define an event
2. Correlation of data of differing modalities
3. Integrating data from heterogeneous data
models
4. Discovery of relevant data sources
5. Presentation and control of information
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
6
Ontology Network:
Reconcile Terminology
Upper
DOLCE
UltraLite
SSG4Env
infrastructure
SSN
SWEET
Service
Schema
External
FOAF
Ordnance
Survey
Flood domain
Role
7 April 2011
Coastal
Defences
Additional
Regions
Data integration on the SemSorWeb – Alasdair J G Gray
7
SemSorWeb Architecture
Applications
Application
Services
Semantic
Registry
Semantic
Integrator
Data Source
Connectivity
Bridge
7 April 2011
Application Tier
Concrete
Resource
Middleware Tier
Data Tier
8
DATA SOURCES
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
9
Channel Coastal Observatory:
Observing Sea-State
43 sensors deployed
around UK coast
• Measuring
•
•
•
•
– Tides (7)
– Waves (24)
– Weather conditions (12)
On and off shore
Bespoke hardware
Fixed functionality
Fixed (low) data rate
– No bursts
• Central data centre
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
10
Publishing CCO Data
• Relational data streams
– envdata_sandownpier_tide (
ts:int, Observed:float, Tz:float,
Hs:float, HMax:float, Tp:float)
• Streaming data service
• Semantically annotated metadata
description
– Service description
– Spatiotemporal data coverage
– Thematic data coverage
• Tide height
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
11
Environment Agency: Flood
Defences Database (NFCDD)
• Flood defence
details
– Type of defence
• Wall
• Sand banks
•…
– Location
– Height
–…
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
12
Publishing NFCDD Data
• Relational database
– locations (id:int, latitude:decimal,
longitude:decimal,location:string,
storm_threshold:decimal)
• Stored data service
• Semantically annotated metadata
description
– Service description
– Spatiotemporal data coverage
– Thematic data coverage
• Sea defences
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
13
INTEGRATING DATA
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
14
Abstract Problem
Integrator
Stored data
service
Streaming
data service
Streaming
data service
Stored data
Sensor
Network
Sensor
Network
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
15
Types of Heterogeneity
Query
Data
capabilities
semantics
stream
source
access
Integrator
Stored data
service
Streaming
data service
Streaming
data service
Stored data
Sensor
Network
Sensor
Network
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
16
Querying Approach
• Use ontologies as common model
Requires:
– Representation of RDF stream
– Expressing continuous queries over an
RDF stream
– Establishing mappings between
ontology models and data source
schemas
– Accessing data sources through queries
over ontology model
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
17
RDF Stream
• Named graph
• Continuously updating
• Triples annotated with timestamp
cd:Observation
cd:observationResult
xsd:double
7 April 2011
STREAM http://www.semsorgrid4env.eu/ccometeo.srdf
...
...
( <ssg4e:Obs1, rdf:type, cd:Observation>, ti ),
( <ssg4e:Obs1, cd:observationResult, “34.5”>, ti ),
( <ssg4e:Obs2, rdf:type, cd:Observation>, ti+1 ),
( <ssg4e:Obs2, cd:observationResult,”20.3”>, ti+1 ),
...
...
Data integration on the SemSorWeb – Alasdair J G Gray
18
SPARQLStream
“Every 5 minutes give me with the wind speed observations
over the last minute in the Solent Region ”
cd:Observation
cd:observationResult
cd:observedProperty
xsd:double
cd:Property
cd:featureOfInterest
cd:Feature
cd:locatedInRegion
cd:Region
7 April 2011
PREFIX cd:
<http://www.semsorgrid4env.eu/ontologies/CoastalDefences.owl#>
PREFIX sb: <http://www.w3.org/2009/SSNXG/Ontologies/SensorBasis.owl#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
RSTREAM
SELECT ?windspeed ?windts
FROM STREAM <http://www.semsorgrid4env.eu/ccometeo.srdf>
[ NOW – 1 MINUTE TO NOW STEP 5 MINUTES ]
WHERE {
?WindObs a cd:Observation;
cd:observationResult ?windspeed;
cd:observationResultTime ?windts;
cd:observedProperty ?windProperty;
cd:featureOfInterest ?windFeature.
?windFeature a cd:Feature;
cd:locatedInRegion cd:SolentCCO.
?windProperty a cd:WindSpeed.
}
Data integration on the SemSorWeb – Alasdair J G Gray
19
Semantic Integrator
SNEEql query
over source
schema
SPARQLStream
query over
ontology
Semantic Integrator
Q
Query
Translator
SNEE Query
Evaluation
Engine
[[q]]
[[Q]]
Tuples
Tuples
q
S2O
Mappings
Streaming
Source
Stored data
Tuples
Tuples
Streaming
Source
Data
Translator
SPARQL bound
variables
7 April 2011
Tuples
Data integration on the SemSorWeb – Alasdair J G Gray
Stored data
20
Query Translation
RSTREAM
SELECT ?x
FROM STREAM <http://…>
[NOW–1 MIN TO NOW
STEP 5 MIN]
WHERE
{
?y a cd:Observation;
cd:observationResult ?x.
}
• Conjunctive queries
()
(
)
(
q x ¬ c1 x1, y1 Ù… Ùcn xn , yn
• Extended with
windows
()
(c ( x , y ) Ù…
)
q x [ t s , te , d ] ¬
1
1
1
(
Ùcn xn , yn
)) [ t , t , d ]
s
e
• Extended with
window-to-stream
()
((c ( x , y ) Ù…
q x ¬ RSTREAM
1
7 April 2011
1
1
(
Ùcn xn , yn
Data integration on the SemSorWeb – Alasdair J G Gray
)) [ t , t , d ])
s
e
21
Query Execution
Ontologies
envdata_westbay
Feature
Observation
hasObservatio
nResult
observe
dPropert
y
xsd:float
locatedIn
Region
Streams
v
v
envdata_hornsea
v
envdata_rhylflats
v
Timestamp: long
envdata_chesil
envdata_milford
S2 O
Mapping
Region
Hs : float
Lon: float
Lat: float
WaveHeigh
tProperty
SPARQLStream
PREFIX cd:
<http://www.semsorgrid4env.eu/ontologies/CoastalDefences.owl#>
PREFIX sb: <http://www.w3.org/2009/SSNXG/Ontologies/SensorBasis.owl#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
SELECT ?waveheight ?wavets ?lat ?lon
FROM STREAM <http://www.semsorgrid4env/ccometeo.srdf>
WHERE
{
?WaveObs a cd:Observation;
cd:observationResult ?waveheight;
cd:observationResultTime ?wavets;
cd:observationResultLatitude ?lat;
cd:observationResultLongitude ?lon;
cd:observedProperty ?waveProperty;
cd:featureOfInterest ?waveFeature.
?waveFeature a cd:Feature;
cd:locatedInRegion cd:SouthEastEnglandCCO.
?waveProperty a cd:WaveHeight.
}
7 April 2011
SNEEql
(SELECT
(SELECT
(SELECT
(SELECT
(SELECT
(SELECT
(SELECT
Lon,timestamp,Hs,Lat
Lon,timestamp,Hs,Lat
Lon,timestamp,Hs,Lat
Lon,timestamp,Hs,Lat
Lon,timestamp,Hs,Lat
Lon,timestamp,Hs,Lat
Lon,timestamp,Hs,Lat
FROM
FROM
FROM
FROM
FROM
FROM
FROM
envdata_rhylflats) UNION
envdata_hornsea) UNION
envdata_milford) UNION
envdata_chesil) UNION
envdata_perranporth) UNION
envdata_westbay) UNION
envdata_pevenseybay)
Data integration on the SemSorWeb – Alasdair J G Gray
22
Data Translation
waveheight
wavets
4.850
1272588663
2.1230
1272587400
<ns9:sparql>
<ns9:head>
<ns9:variable name="waveheight"/>
<ns9:variable name="wavets"/>
</ns9:head>
<ns9:results>
<ns9:result>
<ns9:binding name="waveheight">
<ns9:literal datatype=xs:double>4.850</ns9:literal>
</ns9:binding>
<ns9:binding name="wavets">
<ns9:literal datatype=xs:long>1272588663</ns9:literal>
</ns9:binding>
</ns9:result>
<ns9:result>
<ns9:binding name="waveheight">
<ns9:literal datatype=xs:double>2.1230</ns9:literal>
</ns9:binding>
<ns9:binding name="wavets">
<ns9:literal datatype=xs:long>1272587400</ns9:literal>
</ns9:binding>
</ns9:result>
</ns9:results>
</ns9:sparql>
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
23
FLOOD APPLICATION
“Smashing it all together”
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
24
Flood Web Application
• Discover relevant sources
– Role
– Task
– Location
• Mashup data
– Layers on a map
• Limited execution environment
– Application services provide REST
access to Web services
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
25
Relevant Data Sources
Sensor/Streaming Data
Contextual Data
• Sea-state sensor data
•
•
•
•
•
– CCO
– Port authority
– WaveNet
• Shipping
– AIS
Base Maps
Flood defense details
Land usage
Weather forecasts
Flooding forecasts
• Traffic Alerts
– RSS feed
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
26
DEMO
http://www.semsorgrid4env.eu/services/dynamic-demo
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
27
Conclusions: SemSorWeb
1. Accurate characterisation of conditions that
define an event
–
Declarative queries
–
Query evaluation over streaming and stored
–
Ontology-based access to streaming data
–
Semantic registry (not covered in this talk)
–
Application service to support, e.g. smash-ups
2. Correlation of data of differing modalities
3. Integrating data from heterogeneous data
models
4. Discovery of relevant data sources
5. Presentation and control of information
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
32
Conclusions
• Query-based access to RDF streams
– SPARQLStream
• Continuous query language
– S2O mappings
• Relational streams to ontology
• Global-as-View approach
– Query translation to SNEEql
• Query substition
– Query execution by SNEE-DQP
7 April 2011
Data integration on the SemSorWeb – Alasdair J G Gray
33
Acknowledgements
For more information: http://www.semsorgrid4env.eu/
•
•
•
•
Christian Brenninkmeijer
Alvaro Fernandes
Ixent Galpin
Norman Paton
7 April 2011
• Jean-Paul Calbimonte
• Oscar Corcho
Data integration on the SemSorWeb – Alasdair J G Gray
34
References
1.
2.
3.
7 April 2011
I. Galpin, C. Y. A. Brenninkmeijer, A. J. G. Gray, F. Jabeen,
A. A. A. Fernandes, and N. W. Paton. SNEE: A query
processor for wireless sensor networks. Distributed and
Parallel Databases, 29(1-2):31–85, 2011. Special issue on
Query Processing in Sensor Networks.
A. J. G. Gray, R. García-Castro, K. Kyzirakos,
M. Karpathiotakis, J.-P. Calbimonte, K. Page, J. Sadler,
A. Frazer, I. Galpin, A. A. A. Fernandes, N. W. Paton,
O. Corcho, M. Koubarakis, D. De Roure, K. Martinez, and
A. Gómez-Pérez. A semantically enabled service
architecture for mashups over streaming and stored data.
In European Semantic Web Conference (ESWC 2011),
pages 300–314, 2011. To appear.
J.-P. Calbimonte, Ó. Corcho, and A. J. G. Gray. Enabling
ontology-based access to streaming data sources. In
Proceedings of 9th International Semantic Web Conference
(ISWC 2010), volume 6496 of LNCS, pages 96–111,
Shanghai, China, November 2010. Springer. (Alphabetical
authorship, equal responsibility).
Data integration on the SemSorWeb – Alasdair J G Gray
35