www.derilinx.com
Auditing Our Data
Method & Toolkit
Deirdre Lee
deirdre@derilinx.com
@deirdrelee
11th Feb 2015
“Just as oil was likened to black gold, data
takes on a new importance and value in the
digital age.”
-Commissioner Neelie Kroes
2
www.derilinx.com
Open Data brings many benefits…
3
… where do we begin?
www.derilinx.com
4
https://www.gov.uk/government/publications/open-data-charter/g8-open-datacharter-and-technical-annex
Photo: http://www.telegraph.co.uk/news/worldnews/g8/10128266/G8-Open-DataCharter-why-it-matters.html
www.derilinx.com
What data do we hold?
How is data managed?
What data should we publish?
5
www.derilinx.com
May 2013: Executive Order -- Making Open and
Machine Readable the New Default for
Government Information
In the (UK) government response to the
Shakespeare Review of June 2013, the
government sets out its aim to create a National
Information Infrastructure (NII).
6
www.derilinx.com
What is Data?
7
www.derilinx.com
Data Lifecycle
8
Knud Möller. Lifecycle models of data-centric systems and domains: The abstract data
lifecycle model. 2012. Semantic Web 4.1: 67-88. URL: http://www.semantic-webjournal.net/content/lifecycle-models-data-centric-systems-and-domains
www.derilinx.com
Challenges of Data Management
• Who owns the data?
• Who is responsible for the data?
• What if data is aggregated from multiple
sources?
• How to keep data up-to-date?
• ..etc.
9
www.derilinx.com
Data Asset Framework
10
Ekmekcioglu, C. et al., 2009. Data Asset Framework Implementation Guide, Digital
Curation Centre. Available at: http://www.data-audit.eu/.
www.derilinx.com
COMSODE Methodology for Publishing Datasets as Open Data
11
Nečaský, M. et al., 2015. D5.1 Methodology for publishing datasets as open data,
COMSODE. Available at: http://www.comsode.eu/index.php/deliverables/.
www.derilinx.com
Development of Open Data Publication Plan
• Identify datasets managed/held by organisation.
• Record basic information for each dataset
• Identify datasets to publish as Open Data
• Define how and how often data should be published
• Estimate the effort of publication
• Prioritise data to publish as Open Data
12
www.derilinx.com
D/PER Open Data Audit Template
13
www.derilinx.com
14
www.derilinx.com
Administrative Data
Operational Data
15
www.derilinx.com
Identifying Datasets
• Machine-readable data, not graphs, reports,
applications, or web portals.
• A dataset should be managed and maintained
within the organisation.
• High-value data should be prioritised
16
www.derilinx.com
Machine-Readable Data



17
Table, graph and map from The Quality of Bathing Water in Ireland – An Overview for the
Year 2012
https://www.epa.ie/pubs/reports/water/bathing/thequalityofbathingwaterinireland2012.
html#.U0FxMVdgzvY
www.derilinx.com
High-value Data
• used to increase agency accountability and
responsiveness;
• improve public knowledge of the agency and its
operations;
• further the core mission of the agency;
• create economic opportunity; or
• respond to need and demand as identified through
public consultation
(The U.S. Office of Management and Budget, 2009)
www.derilinx.com
18
Tips on Identifying High-Value Data
• Reports, surveys or other public documents that have
been published by the organisation, or
• Data requests that have been received by the
organisation, e.g. ePQs or FOI requests.
• Data behind Key Performance Indicators, e.g. those used
in IrelandStat
19
www.derilinx.com
Development of Open Data Publication Plan
• Identify datasets managed/held by organisation.
• Record basic information for each dataset
• Identify datasets to publish as Open Data
• Define how and how often data should be published
• Estimate the effort of publication
• Prioritise data to publish as Open Data
20
www.derilinx.com
Metadata
21
www.derilinx.com
22
www.derilinx.com
Development of Open Data Publication Plan
• Identify datasets managed/held by organisation.
• Record basic information for each dataset
• Identify datasets to publish as Open Data
• Define how and how often data should be published
• Estimate the effort of publication
• Prioritise data to publish as Open Data
23
www.derilinx.com
Chain of economic effects of lowered PSI re-use charges
24
DeVries, M. & Hittmair, G., 2014. Open data is fact now- when does the reuse. In
SharePSI Lisbon. Lisbon, PT. Available at: http://www.w3.org/2013/sharepsi/workshop/lisbon/agenda.
www.derilinx.com
Chain of economic effects of lowered PSI re-use charges
25
DeVries, M. & Hittmair, G., 2014. Open data is fact now- when does the reuse. In
SharePSI Lisbon. Lisbon, PT. Available at: http://www.w3.org/2013/sharepsi/workshop/lisbon/agenda.
www.derilinx.com
Chain of economic effects of lowered PSI re-use charges
26
DeVries, M. & Hittmair, G., 2014. Open data is fact now- when does the reuse. In
SharePSI Lisbon. Lisbon, PT. Available at: http://www.w3.org/2013/sharepsi/workshop/lisbon/agenda.
www.derilinx.com
Open Data
--------------------Closed Data
• Personal Data
• Secure Data
27
www.derilinx.com
Upcoming Opportunities
• Open Data Workshop, Thurs 19th Feb
• Open Data Training Day, early March
• Support for internal data audits
• Support for development of Open Data Strategies
28
www.derilinx.com