Scotland's Environment Web
Data Journey 2011-2015
Dave Watson, Duncan Taylor
Session Outline
• SEWeb data journey
– What has been encountered on that journey
• SEWeb as a data consumer
– What do we do with the data?
• Five Star/Linked Data
• SEWeb Data – what next?
Data Journey
Scotland’s Environment Web - DataSEWeb
Journey
National
Security
Partners
Business
as Usual
Eye on Earth
Data
Consumer
Gemini2, SSDI
WMS
INSPIRE
Partners
Data Download
Data
Service
Visualisation
WFS
Linked Data
Scottish
Government
Digital Stategy
Daughter
Sites
Data
Publication
IPR
Data Protection
Environmental
Data Portal?
SEWeb Brand – Daughter Web Sites
Data at Source
Dataset Progress
• ‘Data at Source’
– 55 WMS consumed by Map Viewer -> 239 Data Layers
– 9 Rest Services consumed by Land Information Search (LIS) -> 39
Data Layers
– 10+?? Non spatial data consumed by Visualisation Tools
• Five Star /Linked Data
– 68 SESO Data, 12 Water (SEPA WFD), 1 Site Conditioning (SNH)
• Data Holdings
– Soils/Aquaculture Daughter Sites
– Project Finder
What do we do with the data?
•
•
•
•
•
Themed spatial maps
Advanced Maps
Visualisation Applications
Task Specific Applications
Linked Data Repository
Themed/Advanced Maps
Task Specific Maps – Land Information Search
Visualisation/Discover Data
Why Linked Data? - 5 Star Model of Open Data
#
##
###
####
#####
Available on the web (whatever format) but with an open licence,
to be Open Data
Available as machine-readable structured data (e.g. excel
instead of image scan of a table)
as (2) plus non-proprietary format (e.g. CSV instead of excel)
All the above plus, Use open standards from W3C (RDF and
SPARQL) to identify things, so that people can point at your stuff
All the above, plus: Link your data to other people’s data to
provide context
http://www.w3.org/DesignIssues/LinkedData.html
Linked Data Four Principles
1. Use URIs as names for things
2. Use HTTP URIs so that people can look up those
names.
3. When someone looks up a URI, provide useful
information, using the standards (RDF*, SPARQL)
4. Include links to other URIs so that they can discover
more things.
http://www.w3.org/DesignIssues/LinkedData.html
State of Environment (SOE) – Linked Data Model
State Of Environement
(Linked Data)
Graph Model
soe:State
Essential|supporting
has
Importance
soe:Topic
SOE
(State of Environment)
has
dataset
dct:Dataset
consistsOf
has
soe:Chapter
describedBy
Metadata
SOE – Implementation
Vocabulary/concept scheme
http://data.sepa.org.uk/def/soe
Trial data
http://data.sepa.org.uk/id/soe/chapters
SOE Data Linkages
SOE Data Linkages
Chapter
Topic
Dataset
SEWEB
SOE Data Linkages
SOE Data Linkages
European Indicator (SOE)
EEA
relates to
Chapter
Topic
=
national
indicator
Dataset
SEWEB
SOE Data Linkages
SOE Data Linkages
European Indicator (SOE)
EEA
relates to
Chapter
Topic
Dataset
feeds
SEWEB
links to
Metadata
Data
Provider
publishes
Data view and
download services
SEWeb Data - What Next?
•
•
•
•
Continued Addition of Datasets
What’s in my Area? – Local Datasets/SEWeb Local
Scottish Government Digital Strategy – Data Portals
Graphical Data Models to support ‘State of
Environment’
• Links to European Data Initiatives
Useful Links
– SEWeb www.environment.scotland.gov.uk
– Scottish Soils http://www.soils-scotland.gov.uk/
– Aquaculture http://aquaculture.scotland.gov.uk/
– Linked Data Lab http://data.sepa.org.uk
– SSDI http://scotgovsdi.edina.ac.uk/srv/en/main.home
– INSPIRE http://inspire.ec.europa.eu/
– Water Classification Visualisation
http://www.environment.scotland.gov.uk/get_interactive/dat
a_visualisation/water_body_classification.aspx
End of Presentation – Workshop Support
Slides Follow
Linked Data Architecture
Consumers
SEPA
Architecture
Bespoke Data Feed
DRIVERS
RDBMS
Relational Data
Repository
Dataset Definition.
Metadata
Datasets.
Related not Relational
Cannot do any subsequent
steps without this
definition. Business needs
to define and prioritorise
Metadata
WMS
WFS
Apps
INSPIRE
File Download
Data Feed Future
Data Ingestion
Other Data
Providers
Citizen Scientists
Organisational,
Eg EA,SG etc
SEPA Stakeholders
Public
Linked Data
Ontologies
Vocabularies
REPORTING
SENSE 2/2015
SOE
Useful Links
– SEWeb www.environment.scotland.gov.uk
– Scottish Soils http://www.soils-scotland.gov.uk/
– Aquaculture http://aquaculture.scotland.gov.uk/
– Linked Data Lab http://data.sepa.org.uk
– SSDI http://scotgovsdi.edina.ac.uk/srv/en/main.home
– INSPIRE http://inspire.ec.europa.eu/
– Water Classification Visualisation
http://www.environment.scotland.gov.uk/get_interactive/dat
a_visualisation/water_body_classification.aspx
SENSE 3 – Schema Relationships
State of Environment Reporting
• Defined by chapters (air, water, land, etc)
• Chapters divided into topics, each with a summary
quality assessment
• Datasets support and inform the assessment of the
topic
• A dataset may be related to more than one topic
• Currently published as static pages
State of Environment Reporting
• Remodel as linked data
• Enable publication of metadata on datasets
• Link to data visualisation and download where
available
• Provide contact details where data not yet
published on line
• Provide support and examples of best practice to
assist publication
SEPA as Data Provider
SEPA Reporting Requirements
Information required at many levels
• Internal – SEPA corporate systems
• National – State of Environment; SEWeb
• European – Directive Reports; INSPIRE
Where we were…
GIS
Applications
Reports
SEPA
Database
EU
Reporting
Website
Many applications
Information
Requests
Publications
Many versions
Many formats
What we decided to do
• Focus on data – not applications
• Identify key reporting datasets
• Define them once
• Use them many times…
• …in many formats
Where we’ve got to
Operational
Database
Reports & Analysis
Defined data
“products”
Consistent
data
Consistent
metadata
GIS
Reporting
Database
Publish
Externally
Intranet
EU Reporting
SEPA Website
SEWeb
Where we’re getting to
Operational
Database
Reports & Analysis
Defined data
“products”
Consistent
data
Consistent
metadata
GIS
Reporting
Database
Publish as
WMS; WFS;
Linked data
Intranet
EU Reporting
Websites (SEPA,
SEWeb,…)
Partners
Public
EU
What’s helped
• Scotland’s Spatial Data Infrastructure –
provided framework and standards for
metadata
• SEWeb – prioritisation of datasets
• Government direction – “digital by default“
• EU reporting frameworks – SEIS, SENSE
What we need now
• Agree to use existing standards and
vocabularies
• Define new ones where appropriate
• Encourage use of common reference systems
• Encourage others to use the data
What we get out of it
• Wider (and cleverer) use of data
• Less bespoke development
• Fewer information requests to deal with
• Publish data once – let everyone else get
on with it
Data Architecture
Single Purpose Apps
Consumers
SEPA
Architecture
Bespoke Data Feed
Single Purpose
Apps
E.g. RBMP
RDBMS
Relational Data
Repository
Dataset Definition.
Metadata
Datasets.
Related not Relational
INSPIRE Service Based Architecture
Consumers
SEPA
Architecture
DRIVERS
RDBMS
Relational Data
Repository
Dataset Definition.
Metadata
Datasets.
Related not Relational
Metadata
WMS
WFS
Applications
Service Data Feed
Cannot do any subsequent
steps without this
definition. Business needs
to define and prioritorise
INSPIRE
Linked Data Architecture
Consumers
SEPA
Architecture
Bespoke Data Feed
DRIVERS
RDBMS
Relational Data
Repository
Dataset Definition.
Metadata
Datasets.
Related not Relational
Cannot do any subsequent
steps without this
definition. Business needs
to define and prioritorise
Metadata
WMS
WFS
Apps
INSPIRE
File Download
Data Feed Future
Data Ingestion
Other Data
Providers
Citizen Scientists
Organisational,
Eg EA,SG etc
SEPA Stakeholders
Public
Linked Data
Ontologies
Vocabularies
REPORTING
SENSE 2/2015
SOE
Linked Data ‘Technology Stack’
Consumers
SEPA
Architecture
Bespoke Data Feed
DRIVERS
RDBMS
Relational Data
Repository
Dataset Definition.
Metadata
Datasets.
Related not Relational
Cannot do any subsequent
steps without this
definition. Business needs
to define and prioritorise
Metadata
WMS
WFS
Apps
INSPIRE
File Download
Data Feed Future
Data Ingestion
Linked Data
REPORTING
SENSE 2/2015
SOE
Ontologies
Vocabularies
Rdf Triple Store
Server
ELDA
Define
Equivalences
Other Data
Providers
JSON
Web Apps
RDF/XML
Mashups
SPARQL
Linked Data
Sites/Uers
TURTLE
“Big Data”
Sites/Uers
csv/tsv
“Traditional”
Sites/Uers
HTML
Web
Developers
Citizen Scientists
Organisational,
Eg EA,SG etc
SEPA Stakeholders
Public
Linked Data
5 Star Model of Open Data
#
##
###
####
#####
Available on the web (whatever format) but with an open licence,
to be Open Data
Available as machine-readable structured data (e.g. excel
instead of image scan of a table)
as (2) plus non-proprietary format (e.g. CSV instead of excel)
All the above plus, Use open standards from W3C (RDF and
SPARQL) to identify things, so that people can point at your stuff
All the above, plus: Link your data to other people’s data to
provide context
http://www.w3.org/DesignIssues/LinkedData.html
What is Linked Data?
• Data in which real-world things are given
addresses on the web (URIs), and data is
published about them in machine-readable
formats.
• Describes a method of publishing structured data
so that it can be interlinked and become more
useful.
• Builds upon standard Web technologies such as
HTTP, RDF and URIs, but rather than using them
to serve web pages for human readers, it extends
them to share information in a way that can be
read automatically by computers.
• Enables data from different sources to be
connected and queried.
Linked Data Four Principles
1. Use URIs as names for things
2. Use HTTP URIs so that people can look up
those names.
3. When someone looks up a URI, provide
useful information, using the standards
(RDF*, SPARQL)
4. Include links to other URIs so that they can
discover more things.
http://www.w3.org/DesignIssues/LinkedData.ht
ml
Operational System
Typical Relational Data Table
Surface Water Bodies
COLUMN NAME
DATA TYPE
MANDATORY
ID
Number
Y
NAME
Varchar2(30)
Y
CATEGORY
Varchar2(15)
N
SUB_BASIN
Varchar2(30)
N
CATCHMENT
Number
N
STATUS
Varchar2(30)
N
Typical Relational Data
ID
NAME
CATEGORY
SUB_BASIN CATCHMENT
STATUS
3001
River Almond (Breich
Water confluence to
Maitland Bridge)
River
Forth
61
Poor
3809
River North Esk
(Source to Penicuik
House)
River
Forth
63
High
100208 Loch Shiel
Lake
Argyll
117
Good
200019 South Arran
Coastal
Clyde
Good
As Linked Data
Surface Water Body 3001
is of category
River
Surface Water Body 3001
is called
River Almond (Breich Water
confluence to Maitland Bridge)
Surface Water Body 3001
is in sub-basin
Forth
Surface Water Body 3001
is in catchment
61
Surface Water Body 3001
has status
Poor
Surface Water Body 200019
is of category
Coastal
Surface Water Body 200019
is called
South Arran
Surface Water Body 200019
is in sub-basin
Clyde
Surface Water Body 200019
has status
Good
As Linked Data
Surface Water Body 3001
is of category
River
Surface Water Body 3001
is called
River Almond (Breich Water
confluence to Maitland Bridge)
Surface Water Body 3001
is in sub-basin
Forth
Surface Water Body 3001
is in catchment
61
Surface Water Body 3001
has status
Poor
Surface Water Body 200019
is of category
Coastal
Surface Water Body 200019
is called
South Arran
Surface Water Body 200019
is in sub-basin
Clyde
Surface Water Body 200019
has status
Good
Surface Water Body 3001
is in local authority
West Lothian
Surface Water Body 3001
is in local authority
City of Edinburgh
Surface Water Body 200019 is in postcode district
KA27
RDF/Triplestore
Subject
Predicate
Object
http://data.sepa.org.uk/id/water/surfac
ewaterbody/3001
rdf:type
http://data.sepa.org.uk/def/water/WaterBody
http://data.sepa.org.uk/id/water/surfac
ewaterbody/3001
rdf:type
http://data.sepa.org.uk/def/water/SurfaceWat
erBody
http://data.sepa.org.uk/id/water/surfac
ewaterbody/3001
rdf:type
http://data.sepa.org.uk/def/water/RiverWater
Body
http://data.sepa.org.uk/id/water/surfac
ewaterbody/3001
rdfs:label
“River Almond (Breich Water confluence to
Maitland Bridge)”
http://data.sepa.org.uk/id/water/surfac
ewaterbody/3001
http://data.sepa.org.uk/def/water
/currentOverallClassification
“Overall status – Poor”
http://data.sepa.org.uk/id/water/surfac
ewaterbody/3001
http://data.sepa.org.uk/def/water
/inCatchment
http://data.sepa.org.uk/id/water/catchment/61
http://data.sepa.org.uk/id/water/catchm
ent/61
http://data.sepa.org.uk/def/water
/surfaceArea
6503
http://data.sepa.org.uk/id/water/catchm
ent/61
http://data.sepa.org.uk/def/water
/catchmentType
“Main River”
http://data.sepa.org.uk/id/water/subbas
indistrict/3
rdfs:label
“Forth”
Non SEPA-SEWeb Linked Data
Examples
• Data.gov.uk.
http://data.gov.uk/linked-data/who-is-doing-what
• EA Bathing Waters
http://environment.data.gov.uk/bwq/explorer/index.html
Ordnance Survey
http://data.ordnancesurvey.co.uk/doc/postcodeunit/EH127AT
• Winnipeg
http://now.winnipeg.ca/
• Legislation
http://www.legislation.gov.uk/