Making Open Data Real:
A Public Consultation
#opendata
#openuk
August 2011
1
Contents
Foreword ................................................................................................................................................. 3
1.
Glossary of key terms ............................................................................................................... 5
2.
Table of abbreviations ............................................................................................................... 7
3.
Executive Summary ................................................................................................................... 8
4.
Vision ..........................................................................................................................................10
5.
Scope and Objectives ..............................................................................................................13
6.
Current landscape and scope for improvement ...............................................................14
7.
The Six Opportunities of Open Data..................................................................................19
8.
Policy Challenge Questions ...................................................................................................22
An Enhanced Right to Data ...................................................................................................23
Setting Open Data standards ................................................................................................26
Corporate and personal responsibility ...............................................................................29
Meaningful Open Data ............................................................................................................31
Government sets the example ..............................................................................................33
Innovation with Open Data ...................................................................................................35
9.
Next Steps .................................................................................................................................37
10.
How to respond during consultation ..................................................................................38
Annex 1 – Best Practice and Emerging Evidence ........................................................................40
Annex 2 – Public Data Principles ...................................................................................................56
2
Foreword
Openness is at the heart of this Government‟s approach. Open
Government and Open Data have the power to transform
absolutely the way government and society work for the better.
Transparency is above politics. It will reveal variation in our society
and our public services, re-establishing individual responsibility and
local accountability for public service professionals.
Over the last year, we have begun the transformation of people‟s
relationship with the state. You can now see how government spends your money, and with
whom. You can see crime data at a level that shows what is happening in your area. You can
see the contracts we sign, and hold ministers, including me, to account. Soon you will be
able to see data on primary care outcomes, the effectiveness of schools and rail timetable
data. And, if you want, you can take this data, or any of the other 6,000 plus datasets
published for free, in machine-readable format on data.gov.uk, and develop a business with it.
We are now taking the concept of „Open Data‟ and exploring an approach that could make
this the operating principle of our public services. In the modern age, accessibility of low
cost Information and Communications Technology (ICT), including cloud computing and high
levels of ICT literacy mean that it is relatively low cost for governments to share the „byproducts‟ of public services with everyone. Data that we would collect or create in
delivering public services will have new and radical applications – and the best way to tap
into the UK‟s tradition of creativity and invention is to give that data away.
There are other benefits to Open Data, in addition to demonstrating accountability and
powering economic growth. It supports the provision of real, effective choice. It gives our
public service professionals comparative data that identifies and encourages excellence, and
then drives up quality and improved outcomes. And it presents the opportunity to give users
of public services more power to self-serve, driving a change in the relationship between
users and providers.
Delivering Open Public Services – and Open Data is key to that – is also about reducing the
administrative burden on these services. Better data actually means less data, and more
openness means fewer Freedom of Information requests for this data and less red-tape.
Our proposed approach is, fundamentally, about creating both „pull‟ (a right to data) and
„push‟ (a presumption of publication). With these forces, we will begin to embed openness
and transparency in how we run government. This consultation seeks your views on these
ideas.
There are some challenges to consider. One will be over the scope of our plans for data,
and we welcome your views on how far we should go. We are proposing and consulting on
an extension to the types of organisation to which our Open Data policy would apply. For
example, where we refer to Public Services, we mean public bodies, and those funded,
commissioned or entrusted by Parliament to provide a service. These are areas for
3
consultation, and we welcome views. Where data about public services is held outside the
public sector, we will work with the service providers to free up that data, as we are already
in the Transport sector.
We must, of course, ensure that privacy is preserved and that personal data is protected. It
is my intention that no personal data will be shared with any third party as part of this
initiative. We will consider this issue in further detail, in particular the use of anonymisation
and pseudonymisation techniques to protect personal data.
There is also a debate, for those datasets where quality may not be high, over how we
should balance the immediate publication of data against seeking to improve quality. Given
the costs of improving quality, our judgement should be that we will publish data of lower
quality in preference to holding it back, while seeking over time to drive up the quality of
that data.
Fundamentally, we want to be open about what we do. Open about what we spend. Open
about how public services work. Open about making them better. And so we propose
reform of the whole of the public sector along open, transparent and accountable lines.
What we are doing is not just a first for Britain; these proposals represent our
determination to be the most ambitious Open Data agenda of any government in the world.
Francis Maude
Minister for Cabinet Office
4
1. Glossary of key terms
These are the definitions of key terms used throughout the consultation document but we
are asking, through this consultation, a question about whether we have got them right.
Term
Description
Dataset
Factual data, structured or unstructured.
In relation to public services, this data will typically have been collected as a
by-product of delivery. This includes, for example, key public datasets about
public services; user satisfaction data; and the performance of providers.
For non-government bodies providing public services, information about
aspects unrelated to the delivery of their public service function are not in
scope.
Information
Interpretation and analysis of data that when presented in context
represents added value, message or meaning.
Open Data
Data which can be freely used, re-used and redistributed by anyone.1
In relation to public services, Open Data means data available under the
terms of the Open Government Licence.2
The presumption is that data about public services will be Open Data. It
may be that some data held in relation to public services is made „available‟,
but is charged for.3
Public
Services
Public services are either provided by public bodies, or providers who have
been funded, commissioned or established by statute to provide a service.4
In this document, we will refer to both these groups as „public bodies‟ and
„public service providers‟ or „providers‟ for brevity.
1 http://www.opendefinition.org/government/
2 The Open Government Licence is a simple set of terms and conditions to enable the free re-use of
government and public sector information, see http://www.nationalarchives.gov.uk/doc/open-governmentlicence/. For organisations which are not public bodies, there is the Creative Commons By Attribution or
other recognised Open Licence.
3 These exceptions are discussed below in the section on „Cost‟.
4 Also see the Open Public Services White Paper:
http://files.openpublicservices.cabinetoffice.gov.uk/OpenPublicServices-WhitePaper.pdf.
5
Questions for consultation
The Government would welcome views on the following:
1. Do the definitions of the key terms go far enough or too far?
2. Where a decision is being taken about whether to make a dataset open, what tests
should be applied?
3. If the costs to publish or release data are not judged to represent value for money,
to what extent should the requestor be required to pay for public services data, and
under what circumstances?
4. How do we get the right balance in relation to the range of organisations (providers
of public services) our policy proposals apply to? What threshold would be
appropriate to determine the range of public services in scope and what key criteria
should inform this?
5. What would be appropriate mechanisms to encourage or ensure publication of data
by public service providers?
6
2. Table of abbreviations
Abbreviation
Description
API
Application Programming Interface
DPA
Data Protection Act
EIR
Environmental Information Regulations
FoIA
Freedom of Information Act
ICO
Information Commissioner‟s Office
ICT
Information and Communications Technologies
OFT
The Office of Fair Trading
OGL
The Open Government Licence
RPSI
Re-use of Public Sector Information Regulations
TNA
The National Archives
7
3. Executive Summary
5
3.1 Open Data and Transparency may be the most powerful levers of 21st century public
policy. They present multiple opportunities offering benefit to the public sector, individuals,
businesses and the UK as a whole. They also create opportunities for the UK on an
international stage.
3.2 Open Data enables accountability; it improves outcomes and productivity in key services
through informed comparison; it transforms social relationships – empowering individuals
and communities; and it drives dynamic economic growth. Delivering Open Public Services,
as described in the Open Public Services White Paper – and Open Data is key to that – is
also about reducing the administrative burden on these services. Better data actually means
less but higher quality data, and more openness means fewer Freedom of Information Act
requests for this data and less red-tape.
3.3 Much has already been achieved in opening up the public sector, as demonstrated in the
two open letters from the Prime Minister on transparency. 6 These covered the opening of
finance data, health data, information about public servants, rail timetables, and school
performance. This document will set out proposals for embedding a culture of openness and
transparency in public services. It sets out how we might create the „pull‟ (a right to data)
and the „push‟ (a presumption of publication) that will underpin the further development of
Open Government in the UK.7 This consultation invites views on:
How we might enhance a „right to data‟, establishing stronger rights for individuals,
businesses and other actors to obtain data from public bodies and about public
services;
How to set transparency standards that enforce this right to data;
How public bodies and providers of public services might be held to account for
delivering Open Data;
How we might ensure collection and publication of the most useful data;
How we might make the internal workings of government and the public sector
more open;
5 In this document, „Transparency‟ and „Open Data‟ are at times used interchangeably. Broadly speaking, we
consider Transparency to be the goal and Open Data to be the tool.
6 2010: http://www.number10.gov.uk/news/letter-to-government-departments-on-opening-up-data/; 2011:
http://www.number10.gov.uk/news/letter-to-cabinet-ministers-on-transparency-and-open-data/
7 „Open Government‟ can be broadly defined as a vision for collaborative democracy that uses openness and
modern technology to bring the people‟s expertise together in the policy-making process, see Beth Noveck,
Wiki Government How Technology Can Make Government Better, Democracy Stronger, and Citizens More
Powerful (2009).
8
How far there is a role for government to stimulate enterprise and market making in
the use of Open Data.
3.4 Alongside these proposals, this document sets out a proposed glossary of terms which
establishes the scope and limits of terms like „Open Data‟ and the range of „Public Services‟.
These terms are also open for consultation.
3.5 The document sits alongside a number of other initiatives:
the second phase of the Growth Review, which will include a specific work strand
on the economic opportunities of Open Data;
proposals for the data policy framework of a Public Data Corporation on which a
separate consultation is being launched;
an independent review of privacy and transparency by Dr Kieron O‟Hara of
Southampton University, which is due to be published in the summer; and
the delivery of the commitments outlined in the PM‟s letter on Transparency of
7 July 2011 and revision of existing Information Strategies to be published in
refreshed departmental business plans from April 2012.
3.6 The proposals are designed to promote a collaborative and wide-ranging discussion with
partners in government, the wider public sector, and the broader user community. Over the
summer, the Cabinet Office team responsible for this policy will engage with these
stakeholders to refine and develop proposals further. Following the consultation period and
the conclusion of a full impact assessment, we will revise proposals and publish a White
Paper in the autumn bringing together input from the consultation and other initiatives
outlined above and setting out the way forward.
9
4. Vision
4.1 Information is power and by sharing it, we can deliver modern, personalised and
sustainable public services. Transparency of data in the UK has already transformed our
interaction with the private sector, particularly via the internet. From financial services and
online banking, to travel booking and retailing, access to data has become a means to change
the relationship we have with service providers and retailers: we have access to our
personal data, we compare providers, we exercise choice and we share our feedback.
Online banking was first launched thirteen years ago and now has more than 22m users in
the UK. This is the kind of meaningful Information Revolution that we now seek in our
public services.
4.2 For the public sector, Transparency and Open Data are about helping people find the
right doctor for their needs, or the best teacher for their child, or helping a victim of crime
track whether justice is done. It is about helping frontline professionals compare outcomes
and improve them. It is about giving people access to their individual medical records so
that they can manage their health better and make more informed decisions with their
clinician. It is about giving people the data on local authority spending and delivery that they
need to challenge the value of a service provided. Above all, it is about providing the data
people need to make choices and to help improve public services.
4.3 Open Data may be the most powerful lever of 21st century public policy: it can make
accountability real for citizens; it can improve outcomes and productivity in key services
through informed comparison; it can transform social relationships – empowering individuals
and communities; and it can drive dynamic economic growth. A recent McKinsey report
forecast that the benefit of public data assets to the European economy, if used effectively to
drive innovation and enterprise, could be up to 250bn Euros'.
8
4.4 The proposals in this consultation are designed to support the agenda outlined in the
recently published Open Public Services White Paper. More accessible, higher quality data
can support better quality services that are more responsive to individual and community
needs, giving more freedom and professional discretion to those who deliver them, and
providing better value for taxpayers‟ money. Open Data is central to putting power in the
hands of individuals and local communities to enable people to choose what sort of service
they want and find the best provider to meet their needs.
4.5 In beginning to embed Open Data in the public sector, much has already been achieved:
the public now has access to more than 6,000 data sets through data.gov.uk; the
Government has appointed a Public Sector Transparency Board to support and challenge
public sector bodies in the implementation of Open Data standards; and a new Open
Government Licence has made it easier for public service providers to publish data.
Significant datasets have been released. To name a few:
8 http://www.mckinsey.com/mgi/publications/big_data/index.asp
10
Historic COINS spending data published online
New central government ICT contracts published online
New central government tender documents for contracts over £10,000 published on
a single website, with this information made available to the public free of charge and
new local government contracts and tender documents for expenditure over £500
published in full
New items of central government spending over £25,000 published online and new
items of local government spending over £500 published on a council-by-council basis
Crime data published at a level that allows the public to see what is happening on
their streets
4.6 In his second letter on Transparency published on 7 July 2011, the Prime Minister
announced a series of unprecedented commitments to focus on public service outcomes,
through publication of new key data on the National Health Service, schools, criminal courts
and transport. These will be delivered over the coming year.
4.7 But there is still far more to be done – at present the reality for citizens is that getting
access to meaningful data about their public services can still be difficult and is sometimes
impossible. Equally for enterprise, particularly start-ups and SMEs, getting access to data that
helps grow their business may be difficult or close to impossible. The quality of data that is
currently published is often poor, and publication may be intermittent, which is unhelpful for
business in particular. Standards do not exist across departments or wider public bodies, so
it is difficult to make comparisons. Data may be published without clear explanations of
context, meaning that in reality it is difficult to use. Fundamentally, the right to continued
access to a dataset, once released, does not exist. The culture within the public sector and
with public service providers is not currently focused on making data available.
4.8 This document sets out a number of levers that the Government is considering using to
make Open Data and Transparency the operating principle of public services, including the
creation of an enhanced right to data, giving individuals and organisations the right to access,
interpret and utilise data in an enhanced form for bodies already subject to the Freedom of
Information Act (FoIA) or Environmental Information Regulations (EIR) legislation, and a new
right to data for a wider range of public service providers extending to cover providers who
have been funded, commissioned or established by statute to provide a service.
4.9 Simply stated, the proposals outline how we will move to a position where most data
held by public bodies and about public services will be available for re-use under the Open
Government Licence, except in very specific circumstances. There will continue to be
exceptions, for example for personal data, data that through release might compromise
national security or Ministerial decision-making. When considering whether or not to
charge for data, a transparent business case setting out why will be made, including any cost
or value for money implications, unless the data falls into one of the exceptions set out
above.
11
4.10 It is our intention that data already provided free should not be charged for, and there
is no question of charging for data required to hold public bodies accountable or for the
“key data about public services, user satisfaction and the performance of all providers.
9
4.11 This policy framework is set out at a high level for consultation over the summer. The
Transparency Team in the Cabinet Office will use that consultation to refine and develop
proposals further, and following consultation will publish a fuller document confirming the
Government‟s policy approach.
9 Para 3.4 of the Open Public Services White Paper:
http://files.openpublicservices.cabinetoffice.gov.uk/OpenPublicServices-WhitePaper.pdf.
12
5. Scope and Objectives
5.1 Greater transparency of government potentially offers a transformative effect on UK
public services, civil society and the economy. From enabling more effective accountability
to driving social and economic growth, significant benefits could be realised through three
main types of Open Data:
1. Large, non-personal datasets collected routinely by public services e.g. Hospital
Episode Statistics (HES) on NHS admissions and outpatient appointments in England;
2. Right of individuals to access and control their own service user records e.g. a school
10
or personal health record ; and
3. User feedback on services e.g. comments and suggestions from users of health,
personal or social care services such as iwantgreatcare.org or bestcarehome.co.uk.
5.2 These three main types of Open Data could operate within a dynamic market for
information, made up of many different – but often overlapping – business models.
However, the second category of data described above raises different issues, and is not
considered further in depth further in this paper.
5.3 As the Glossary notes, we define Open Data to mean public datasets which are available
free of charge for re-use under the Open Government Licence, covering all data relating to
the provision of public services. This will include datasets on access to services, user
satisfaction, spending, performance and equality.
5.4 In devolved areas of policy, it is for the devolved administrations to determine their own
public service reforms and we will seek to work in partnership to share best practice.
10 As stated in the section on Privacy, personal data will not be shared with third parties and full consideration
will be given to any release of data that could potentially impact on privacy.
13
6. Current landscape and scope for
improvement
Summary of current environment
6.1 At the moment, government‟s approach to the release of public data can be described as
haphazard and in need of reform. Some key examples include:
It is not always clear what government department or public body from which to
access data, nor the process required (e.g. Freedom of Information requests can be
denied because the relevant department has not been contacted, and
the permission process for licensed information – even where anonymised - can be
lengthy and complicated);
We collect but do not publish other datasets e.g. data behind the indices of
deprivation is not released, and the questions underlying the monthly Labour Force
survey (without additional permissions);
Local public services collect data but do not always make it available in accessible
public formats e.g. release of location identifiers has been restricted for linking with
certain datasets (including Citizenship Survey and Pupil Level School Census data)
impeding more insightful analysis on migration and patterns of demand for education;
and
Boundaries are not clear as to what public service data is. Some public service data is
held by non-government or quasi government service providers and not subject to
FoIA legislation.
11
6.2 This paper proposes to cover data relating to the provision of „public services‟ , and
welcomes views through this consultation.
6.3 The current legislative, statutory and regulatory landscape is multi-faceted and its key
elements, summarised in Table 1 below, include: Freedom of Information Act (FoIA); the
Environmental Information Regulations (EIRs); the Data Protection Act (DPA); the Re-use of
Public Sector Information Regulations (RPSI); and the INSPIRE Regulations. The first three
are regulated by the Information Commissioner‟s Office (ICO) and The National Archives
(TNA) has further regulatory responsibilities, including the investigation of complaints under
the RPSI as well as managing Crown Copyright, and monitoring compliance against required
standards under the Information Fair Trader Scheme (IFTS). For queries and complaints
under the INSPIRE Regulations, the ICO deals with restrictions of access, TNA with
charging, and the UK Location Coordination Unit (DEFRA) with technical matters. Scotland
will make its own arrangements. The degree to which this legislative framework applies to
different public service providers varies, and this has often helped to create confusion and
act as a barrier to openness.
11 In line with the recent Open Public Services White Paper http://www.cabinetoffice.gov.uk/resourcelibrary/open-public-services-white-paper
14
Table 1: Overview of the current right to data landscape
Legislation and regulations
Freedom of Information Act – FoIA gives a statutory right of access to all information held by over
100,000 public authorities, including government departments, local authorities, police and fire services,
schools and universities, and the NHS. It applies to English, Welsh, Northern Irish and UK-wide bodies, but
not to Scottish bodies, which are subject to a separate Scottish Act. There is a presumption of openness
although some information may be deemed exempt from disclosure, for example in order to safeguard
personal data, national security, commercial interests, and the effective conduct of public affairs.
Environmental Information Regulations – The EIRs give the public access rights to environmental
information held by a public authority. Most bodies subject to the FoIA are also subject to the EIRs. The
EIRs also extend to some public authorities not subject to the FoIA. Scotland has its own EIR legislation.
Data Protection Act – The DPA (1998) provides a statutory framework for the „processing‟, i.e.
collection, use and disclosure, of personal information about living individuals. It applies to any organisation
or individual within the UK who decides how to process such personal data. It also provides a number of
rights to individuals, including the right to request to see any data that an organisation might hold on
them. The DPA implements the European Union's Data Protection Directive 95/46 into UK law, which is
currently being reviewed by the European Commission.1
Re-use of Public Sector Information Regulations – These regulations cover a wide range of public
sector bodies in the UK, from central government departments and the devolved administrations, to parish
councils, the health service and the emergency services. The RPSI encourage the re-use of public sector
information by removing obstacles to re-use. Public sector bodies are obliged to provide clear statements
on their arrangements for re-use, including any licence terms and conditions and details of any charges.
INSPIRE – the 2009 UK INSPIRE Regulations transposed the 2007 EU INSPIRE Directive, which aims to
improve environmental policy-making in Europe. Member States are required to make available in a
consistent format spatial datasets within the scope of the Directive, and create services for accessing these
datasets. Doing so will enable datasets to be more easily shared and facilitate the development and
monitoring of environmental policy and practice in Member States and across the European Union.
Implementation of INSPIRE forms part of the UK Location Strategy.
Bodies with regulatory powers
Information Commissioner‟s Office – the Information Commissioner regulates information rights
legislation, specifically DPA, the Privacy and Electronic Communications Regulations, FoIA and the EIRs. The
Commissioner‟s remit is UK-wide except in relation to the Scottish FoIA and the EIRs, for which there is a
separate Scottish Commissioner.
The National Archives – under RPSI, the Office of Public Sector Information (part of TNA) has a
statutory responsibility for the investigation of complaints relating to re-use. This links to the IFTS, which
sets standards and principles, such as simplicity, transparency and fairness, which information providers in
the public sector are required to meet as a condition of accreditation. 1 TNA also has agreements in place
with key regulatory partners, the Office of Fair Trading (OFT), the ICO and the Scottish Information
Commissioner.
15
Opportunities for improvement
6.4 Despite this array of legislation and guidance, potential requesters and re-users of data
face a range of barriers to accessing, using and re-using data that could generate economic
or social value:
a. Cost barriers resulting from historic ICT procurement and data management –
information is held within government in a way that makes it costly to release, so
requesters are refused on the grounds of cost.
b. Contracts agreed without consideration of Open Data principles – situations where
public bodies have not considered in advance how data gathered might be made
open.
c. Formats, quality or timeliness – often it will turn out that data is collected, but the
manner in which it is held makes it difficult to use and re-use.
d. Incorrect application of exemptions or exceptions – which can lead to information
being withheld inappropriately, requiring the use of the appeals process to challenge
the decision.
e. Charging to cover short-term costs – situations where the public sector charges for
information in order to cover short term costs, perhaps not considering the
opportunity costs of lost innovation or more efficient public services, which
additionally would likely to be accrued elsewhere.
f.
Licensing conditions and processes which are restrictive and can stifle innovation these licences may also be delivered through slow, inflexible and cumbersome
processes, where they are not released under the OGL and the wider UK
Government Licensing Framework.
6.5 In moving to the approach outlined in this document, there are a number of
considerations, in particular cost and privacy.
Cost
6.6 Simply stated, the proposals outline how we might move to a position where most data
held by public service providers about the provision of public services would be available for
re-use under the Open Government Licence, except in very specific circumstances. There
are a number of factors that may preclude releasing data for free re-use and these would
create some exceptions to this rule.
6.7 It is our intention that data already provided for free re-use should not be charged for,
and there is no question of charging for data required for holding public bodies accountable
or for “key data about public services, user satisfaction and the performance of all providers
from all sectors. This will include data on user satisfaction, spending, performance and
equality.”12
12 Para 3.4 of the Open Public Services White Paper:
http://files.openpublicservices.cabinetoffice.gov.uk/OpenPublicServices-WhitePaper.pdf.
16
6.8 When considering whether or not to charge for data, a transparent business case setting
out why will be made, including any cost or value for money implications. Under existing
rules, where Crown bodies wish to charge for the re-use of data, they are required to justify
this against the criteria for exceptions for marginal cost pricing to The National Archives,
which manages Crown Copyright and database right. This justification ensures that
government does not limit or restrict re-use.13
6.9 Any charging should follow existing rules in Managing Public Money (MPM) guidance14.
MPM guidance states that much information about public services should be available free, or
at low cost, in the public interest. The guidance explains that where re-use of data is
charged for the norm is to charge at marginal cost and for value-added data and information
currently sold by trading funds the norm is to charge at full cost plus an appropriate rate of
return. Other value-added data services may also be charged for, for example: services
commissioned in response to particular requests; services where there are statutory powers
to charge; and publications processing publicly gathered data for the convenience of the
public, through editing, reclassification or other analysis.
6.10 The norm for all information supplied by trading funds is to charge, within the
constraints of MPM guidance, although the Government is committed to moving to making
more data freely available, within the constraints of affordability and value for money. The
implication for a Public Data Corporation (PDC) is being explored through a public
consultation.
6.11 Over time, we would expect costs to lower in most cases, particularly as the ICTrelated costs of providing data lower. There will also be benefits which may offset costs in
the medium-term from reductions in bureaucracy through transparency of data; reduced
FoIA requests for the data published; and some reduction of data where collection is
considered to be unnecessary. To mitigate costs during a time when the public sector must
be particularly mindful of public funds, we propose that the emphasis be placed on releasing
new data rather than old, and on releasing data „as is‟, rather than spending time and
resource on improving quality immediately.
6.12 In the autumn, when the response to the consultation and full strategy is set out, we
will include a full Impact Assessment of costs and benefits of Open Data, including
consideration of any opportunities and burdens on public bodies and public service providers
which will inform the policy development.
Privacy
6.13 With the exception of allowing individuals access to their own personal records, at no
point in this document does the data we describe refer to personal data. Transparency will
not be extended at the cost of privacy. Releasing greater quantities of anonymised data
brings with it complex questions about how we can ensure that personal data remains
protected. We will consider how we will, practically, ensure that personal data is
13 See http://www.nationalarchives.gov.uk/information-management/ifts/cost-pricing.htm
14 http://www.hm-treasury.gov.uk/psr_mpm_index.htm
17
anonymised, particularly when they are released alongside many other datasets, which have
the potential to be merged.
6.14 The Minister for the Cabinet Office has commissioned an independent review by Dr
Kieron O‟Hara of Southampton University to consider privacy and Transparency,
recommend steps for ensuring that as the Open Data approach is adopted, privacy is not
compromised. We will publish this report over the summer.
6.15 In addition, where there is good reason not to publish information or data as set out in
existing exemptions or exceptions in the legislation, for example because it would
compromise national security – these exemptions will remain.
18
7. The Six Opportunities of Open
Data
Accountability
7.1 Transparency (an older term than „Open Data‟) has often been focused on accountability.
The expectation is that modern, democratic government shares information with the society
it governs to demonstrate freedom from corruption and appropriate use of public funds.
Accountability is an important strand of the Open Data agenda and much of the focus of
policy over the past few years has continued to be on holding politicians and public bodies
better to account. The MPs‟ expenses scandal demonstrated what can go wrong when
systems and processes operate outside the glare of effective public scrutiny. It also revealed
the strength of public appetite to hold politicians and public bodies to account. However,
critical to the current Open Data agenda is the recognition that there are wider benefits to
releasing data created with public funds, beyond demonstrating accountability.
Choice
7.2 Evidence suggests that choice matters to citizens, particularly around how users engage
15
with public services. While many of the public do not associate choice with an ability to
drive up quality standards, the evidence shows that – where it exists – choice can be an
16
effective mechanism for improving standards. The Open Public Services White Paper sets
out a vision for putting people in control, either through direct payments, personal budgets,
entitlements or choice. Providing comparative information enables offering meaningful choice
to become a reality in public services. Equipped with an understanding of variation in
service quality, we can make more informed choices about which services are most
appropriate to us or our family members. At present, it is not easy to compare the quality
of public services. As personal and community budgets extend across a greater number of
public services, individuals and communities will rely upon Open Data and information to
make shared decisions.
15 http://www.ipsosmori.com/DownloadPublication/1345_sri_what_do_people_want_need_and_expect_from_public_services_
110310.pdf
16 http://clients.squareeye.net/uploads/2020/documents/ESRC_Allan%20and%20Burgess_FINAL.pdf
19
Productivity
7.3 Public reporting of costs and comparative outcomes can be a driver of efficiency. The
first conclusion drawn in Phillip Green's Review as to why government conducts business
inefficiently was that: “Data is very poor and often inaccurate.”17 HM Treasury‟s Operational
Efficiency Review noted the need for “consistent, comparable data” for organisations to
know whether the services they deliver constitute good value for money. Internal collection
and monitoring of management information is critical for driving efficiency improvements,
and for making informed strategic decisions.
7.4 At present, where data is not open outside government, it may often not be available
inside government as well. Public sector bodies are not easily able to benchmark their costs
and the quality of their services against their peers and may have falsely high – or low –
understandings of their performance. Healthy competition between service providers
should develop, driving further improvement and minimising duplication and waste.
Quality and Outcomes
7.5 Benchmarking data on comparative costs and quality of services helps to drive up quality
of outputs and outcomes, especially when peer-based competition is sharpened by public
scrutiny. Additionally, the publication of meaningful data can improve user engagement and
even input. For example, access to personal health records could encourage some to take a
more proactive approach to their own health, while access to records can enable parents
and students to engage more closely with the education process. It has already been argued
that making data open incentivises improvements in the quality of that data. High quality data
is a pre-requisite of outcomes-based commissioning, something that is being considered
across a wide range of public services, from welfare-to-work to drug rehabilitation.
Social Growth
7.6 Open Data presents opportunities for public service transformation by giving users more
power to self-serve. Just as the financial services industry has been revolutionised by the
introduction of online banking, so providing wider online access to medical and educational
records will enable service design and delivery to be changed radically, reducing cost and
improving quality.
7.7 Open Data can also create a platform for more informed public debate. This in turn
means the public is better equipped to hold local, and central, government to account.
18
Open Data tools such as Miami 311, police.uk and OpenlyLocal enable citizens to be more
informed about public services in their area.
17 http://download.cabinetoffice.gov.uk/efficiency/sirphilipgreenreview.pdf
18http://www.heygov.com/default.aspx?MapID=Miami311V2; http://police.uk/; http://openlylocal.com/
20
Economic Growth
7.8 Finally, Open Data can be a driver of economic growth. A new market for public service
information will thrive if data is freely available in a standardised format for use and re-use,
particularly in the life sciences; population data mining and risk profiling; consumer
technologies; and media sectors. At present the market for information on public services is
highly underdeveloped. Open Data across government and public services would allow a
market in comparative analytics, information presentation and service improvement to
flourish. This new market will attract talented entrepreneurs and skilled employees, creating
high value-added services for citizens, communities, third sector organisations and public
service providers, developing auxiliary jobs and driving demand for skills.
7.9 For a more detailed exploration of the opportunities and benefits created by Open Data,
please refer to Annex 1, which sets out some of the emerging evidence and best practice
behind Open Data.
21
8. Policy Challenge Questions
8.1 The following section sets out a series of challenging questions in six key areas to start
the debate.
8.2 The six areas identified as presenting opportunities for change, creating both demand for
data and supporting and enabling supply, are:
1. An enhanced right to data: how do we establish stronger rights for individuals,
businesses and other actors to obtain, use and re-use data from public service
providers?
2. Setting transparency standards: what would standards that support an enhanced
right to data among public service providers look like?
3. Corporate and personal responsibility: how would public service providers be
held to account for delivering Open Data through a clear governance and leadership
framework at political, organisational and individual level?
4. Meaningful Open Data: how should we ensure collection and publication of the
most useful data, through an approach that enables public service providers to
understand the value of the data they hold and helps the public at large know what
data is collected?
5. Government sets the example: in what ways could we make the internal
workings of government and the public sector as open as possible?
6. Innovation with Open Data: to what extent is there a role for government to
stimulate enterprise and market making in the use of Open Data?
8.3 This document provides a high-level framework of the areas for consultation over the
next three months. The Cabinet Office will review responses and firm up proposals in a
White Paper in the autumn with appropriate impact assessments. In some areas, the debate
through this consultation will support other planned programmes of work such as the postlegislative scrutiny of FoIA.
22
An Enhanced Right to Data
How would we ensure citizens have a legally enforceable right to access easily, use and re-use data
held about any publicly-funded service?
8.4 Existing legislation provides access to and re-use of a
wide range of public sector data but, as described
above, access can be limited in a number of ways. The
legislation does not extend to a large number of bodies
who provide public services and receive public funds.
FoIA is due to undergo post-legislative scrutiny from
autumn 2011, while negotiations on a new Data
Protection Instrument, which are likely to lead to a
revised DPA, are due to start later this year. A review
of the scope of the Directive on the Re-use of Public
Sector Information is also being undertaken by the
European Commission.
"The point of open
information is not merely to
expose the world but to
change it...These days
democratic openness means
more than that citizens can
vote at regular intervals in
free and fair elections. They
also expect to have access to
government data."
The Economist
8.5 This document does not seek to anticipate these
reviews, but seeks views on how the existing
framework could be improved or built upon to support
the agenda. The key policy challenge question is how can we establish a stronger
presumption in favour of the publication of data than that which currently exists? How do
we ensure that valuable data and data which serves the public interest is more readily
available from a wider range of bodies?
8.6 Would change in the following areas assist in this aim?
1) Embedding the principle that data should be open by default in existing
legislation. Where exemptions or exceptions do not apply, the presumption will be that
the data will be published.
2) Introducing a new requirement that all public bodies and providers of public
service proactively publish data about the services they deliver.
3) Establishing an enhanced right of challenge against decisions not to publish
data to an independent body, accompanied by a power for that body to order
not just the release of public datasets, but also the format, quality and regularity
of publication. This power should be strong but infrequently used, and the body should
ensure that local management are only over-ruled when all other avenues have been fully
explored.
4) Exploring whether the current fees regulations and cost limits under the FoIA
should be amended to facilitate the release of more data. At present, public service
providers subject to the FoIA can refuse to release any information where it would take
more than 18 hours (24 hours for government departments) to locate, retrieve and extract
that information. This time limit does not include the time taken to consider whether the
23
information is exempt from release or to prepare it for release. The fees regulations only
apply to requests for information. Public bodies have discretion about whether they
proactively publish datasets (i.e. not in response to requests). Consideration of this issue
would be likely to include:
Should there be a higher cost limit for datasets to that provided for other
information under the FoIA so that more are released? If so, what should this be?
Would the additional resource required be proportionate to the aim of increasing
transparency?
Should a public service provider be required to pay some of the cost so as not to
create an incentive to inflate cost estimates? Does the Information Commissioner
provide an adequate avenue of address where costs are miscalculated?
If proactive publication of datasets were mandatory, should a public service provider
be able to refuse to publish certain datasets on the grounds of cost – if so what
would be an unreasonable cost?
Should a public service provider be required to publish datasets over the cost limit if
the cost is met by the requester? Are there circumstances where this may prevent
the public service provider from delivering its core functions?
Should there be a new, higher cost limit for FoIA requests for data held within ICT
systems procured after July 2012? Raising the cost limit would provide an incentive
for public service providers to give due weight to the importance of designing
systems from which data can be extracted quickly.
5) Exploring whether the Information Commissioner has sufficient powers to
enforce the legislation. The Information Commissioner has a number of enforcement
powers in relation to FoIA and EIRs. These include the ability to issue Decision Notices
requiring the release of information, Enforcement Notices requiring a public service provider
to take specific steps, and Information Notices, requiring that the public service provider
provide the Commissioner with specific information. The Commissioner also has powers of
entry and inspection in specified circumstances. It is an offence under FoIA and DPA to
alter, deface, block, erase, destroy, or conceal information with the intention of preventing
its disclosure. This offence can apply to any individual. Are these powers sufficient to
enforce an enhanced right to data?
6) Legislating to provide statutory time limits for internal reviews. At present
there is no statutory time limit for the completion of internal reviews by a public service
provider under the FoIA and this can lead to delays in the provision of information. The ICO
recommends that internal reviews should be completed within 20 working days, or 40
working days in exceptional circumstances. The EIRs require that an internal review must
be completed as soon as possible and within 40 working days.
7) Ensuring through procurement rules that data collected by public service
providers is stored in ICT systems that minimise the cost and difficulty of
24
publishing data online. Currently, data requests are often refused because the data is
stored in a fashion that makes it difficult to extract. Procurement rules for ICT systems
could be reformed to ensure that new systems are designed in ways that make data
extraction easier and cheaper.
8) Mandating a phased introduction of „Open by Default', delivered through a
new generation of ICT systems, and accompanying policies. A true culture-shift to
Open Data will require more than new powers. It will require that public sector ICT
systems make 'Open by Default' the most attractive option for procurement.
8.7 This ultimately means authoring data in tools that are 'online by default'. It is only once it
is easier to publish data on the internet than it is to store it in local files, or on paper, that
the public sector can be expected to adopt a more open model. This will mean replacing
outdated data management systems not with more modern versions of what was had before,
but fundamentally new tools that end the classic model of saving files to network drives.
Such tools will largely be delivered by the private sector, but the public sector will have a
role to play in being an intelligent, demanding customer, and developing parts of the
infrastructure itself when necessary. This should all be delivered through a medium term
plan for changing the way in which government ICT systems are procured, which is mostly
out of the scope of this consultation.
Questions for consultation
The Government would welcome views on the following:
1. How would we establish a stronger presumption in favour of publication than that which
currently exists?
2. Is providing an independent body, such as the Information Commissioner, with enhanced
powers and scope the most effective option for safeguarding a right to access and a right
to data?
3. Are existing safeguards to protect personal data and privacy measures adequate to
regulate the Open Data agenda?
4. What might the resource implications of an enhanced right to data be for those bodies
within its scope? How do we ensure that any additional burden is proportionate to this
aim?
5. How will we ensure that Open Data standards are embedded in new ICT contracts?
25
Setting Open Data standards
There should a presumption of high quality publication for all data that is created with public funds
and government must be held to account for meeting that standard.
8.8 There is a strong need to bring about a behavioural
and cultural change within public service providers leading
to „open by default‟ being embraced. In the short-term
there is a debate, for those datasets where quality may
not be high, over how we should balance the immediate
publication of data against seeking to improve quality.
Given the costs of improving quality, our judgement is
that we should publish data of lower quality in preference
to holding it back, whilst seeking over time to drive up the
quality of that data.
8.9 The key policy challenge question is how to ensure
that an enhanced right to data is brought to life and made
real? Should this be through:
“Lack of common standards
…makes it excruciatingly
time consuming to exploit
the information.”
Literature review for EFMC
2011 (European Federation
for Medicinal Chemistry):
AstraZeneca, Royal Society
of Chemistry, Pfizer UK and
others
formalising, through a Code of Practice or opt-in
19
process, the Public Data Principles articulated by the Public Sector Transparency
Board;
making clear the minimum that citizens can expect on publication and quality of data,
which will include compliance with the Public Data Principles;
ensuring a line of continuous improvement for public service providers in achieving
the highest ratings for their published data when compared against the Five Star
Rating20 for Open Data:
o
“★ Available on the web (whatever format), but with an open licence
o
★★ As (one star) plus available as machine-readable structured data (e.g. Excel instead
of image scan of a table)
o
★★★ As (two star) plus use non-proprietary format (e.g. CSV and XML)
o
★★★★ All the above plus, use open standards from the World Wide Web
Consortium (W3C) such as RDF and SPARQL21 to identify things, so that people can
point at your stuff
19 http://data.gov.uk/wiki/Public_Data_Principles. Also at Annex 2 of this document.
20 Developed by Sir Tim Berners-Lee, see http://www.w3.org/DesignIssues/LinkedData.html
21 Resource Description Framework (RDF) is a family of World Wide Web Consortium (W3C) specifications
originally designed as a metadata data model. It has come to be used as a general method for conceptual
description or modeling of information that is implemented in web resources, using a variety of syntax
formats; SPARQL Protocol and RDF Query Language is an RDF query language.
26
o ★★★★★ All the above, plus link your data to other people‟s data to provide
context.”
setting out how citizens can challenge where there is failure in the process (although
we expect the public will rarely need to revert to this because data will be
proactively published);
having in place an Open Data compliance monitoring process which outlines how,
when and where public service providers should report their progress; and
establishing an obligation to consider and, if appropriate, act on user feedback – even
where it has been collected independently of the public body or public service
22
provider?
8.10 We will work with data providers and the data re-user community through data.gov.uk
to set standards. We will set general standards for data release, which will cover policy and
technical measures so data can be used as widely as possible. For specific datasets, like spend
or staffing data, we will also set data definitions, to explain exactly what the data mean, so
that they can be combined and compared for re-use. Together, these will:
make clear that, with very narrow restrictions, licences must cover free, commercial
23
re-use with public service providers not normally selling data. We will build on the
successful OGL, which makes re-use of Crown Copyright and Crown Database
material free for commercial and non-commercial purposes, to create one or more
licences which will be prescribed for public bodies where they are making datasets
available for re-use. In most cases, the expectation will be that this licence will be the
OGL;
merge information asset registers, publication schemes and other data lists over time
into a single data inventory, alongside which would sit the „unlocking service‟ that
provides for citizens and business to make request for datasets not currently
published or planned to be published;
encourage continuous improvement by adoption of recommended publication
formats appropriate to the context;
set consistent expectations of the appropriate quality of meta-data; and
for standardised data co-ordinated across government, set the definitions of the data
to be provided and their context.
22 The Open Public Services White Paper published on 11 July 2011 is consulting on how this can best be
achieved in each of the individual services, including looking at how to collect performance and customer
satisfaction data from service providers, and whether this should be collected in a standardised form to
enable comparison and transparency: http://www.openpublicservices.cabinetoffice.gov.uk/
23 The norm for all information supplied by trading funds is to charge, within the constraints of Managing Public
Money guidance, although the Government is committed to moving to making more data freely available,
within the constraints of affordability and value for money. The implication for a Public Data Corporation
(PDC) is being explored through a public consultation.
27
8.11 Statisticians in the Government Statistical Service (GSS) will play an important role in
the delivery of the Open Data agenda. There are obviously close links between official
statistics and Open Data with many issues in common. The GSS has done much to ensure a
consistent application of standards across departments on which we can build further. Over
the coming months, we will explore with the GSS how to ensure coherence in
implementation between the Transparency agenda and the Code of Practice for Official
Statistics which guides the quality of the statistics they produce.
Questions for consultation
The Government would welcome views on the following:
1. What is the best way to achieve compliance on high and common standards to allow
usability and interoperability?
2. Is there a role for government to establish consistent standards for collecting user
experience across public services?
3. Should we consider a scheme for accreditation of information intermediaries, and if
so how might that best work?
28
Corporate and personal responsibility
Public services must show leadership in making itself as open as possible.
8.12 It is essential for the sustainability of the Open Data agenda that information and data
governance is embedded in public service culture. The
key policy challenge question is how we can ensure that
public service providers in their day-to-day decision"Without strong
making honour the commitment to Open Data, while
leadership, transparency of
ensuring that personal data is fully protected and
decision making and
respecting security considerations. Should this be
effective and intelligent
through:
accountability, trust can [sic]
Introducing a corporate responsibility at Boardlevel to ensure that the right to data is being met
(for the organisation and all service providers in
the public, private and third sectors) based on the
24
be abused and confidence
damaged."
Chair, Committee on
Standards in Public Life
Caldicott Guardian model ;
Strengthening the role and broadening the
membership of the Public Sector Transparency Board chaired by the Minister for
Cabinet Office;
25
Bringing the Sector Transparency Board model to other parts of the public sector
holding datasets of greatest value. These will bring input from experts to support and
challenge government in making more data public. These boards could ensure that
data publication is prioritised to deliver the maximum benefit; and
Reviewing the existing governance and regulatory model for public sector
information in government. Is the existing framework too fragmented, given – for
example – policy responsibilities lie across various bodies including the Ministry of
Justice, Cabinet Office and The National Archives?
24 In the Health context, a Caldicott Guardian is a senior person responsible for protecting the confidentiality
of patient and service-user information and enabling appropriate information-sharing. The Guardian plays a
key role in ensuring that the NHS, Councils with Social Services responsibilities and partner organisations
satisfy the highest practicable standards for handling personal patient information.
25 The Local Public Data Panel provides a challenge and support function for local government datasets:
http://data.gov.uk/blog/local-public-data-panel. In the PM‟s Letter on transparency of 7 July, we have
committed to establish a Sector Board model for Health, Education, Transport, Crime and Justice and
Welfare: http://www.number10.gov.uk/news/letter-to-cabinet-ministers-on-transparency-and-open-data/
29
Questions for consultation
The Government would welcome views on the following:
1. How would we ensure that public service providers in their day to day decisionmaking honour a commitment to Open Data, while respecting privacy and security
considerations.
2. What could personal responsibility at Board-level do to ensure the right to data is
being met include? Should the same person be responsible for ensuring that personal
data is properly protected and that privacy issues are met?
3. Would we need to have a sanctions framework to enforce a right to data?
4. What other sectors would benefit from having a dedicated Sector Transparency
Board?
30
Meaningful Open Data
Public services should only collect and publish data that is meaningful and useful and stop collecting
data that has no value.
8.13 A right to data is meaningless without knowledge
of what is available.
8.14 The public sector currently collates numerous
information and data lists, including FoIA publication
schemes, departmental information strategies, both
general and personal information asset registers, and
the Local Government Data list. This leads to
unnecessary bureaucracy, lack of democratic
accountability and a disparate view of what
information exists.
8.15 The key policy question concerns how we would
provide potential users with a clear understanding of
what information currently exists and what can, will
or cannot be accessed and re-used. Should this be
through:
“Once medical researchers
start publishing their data,
and depositing it in data
archives, they will discover
not only that it is painless,
but that it affords huge
advantages to medical
science, and to patients
present and future.”
Medical Research Scientist,
New York (BMJ editorial)
Establishing a framework for public service providers to have common, consistent and
transparent data inventories outlining what datasets are held, and whether they are
open or not, using standards set by central government. Inventories would need to be
built in a modular way, over time, and should begin with high priority data;
Developing a clear methodology to support intelligent inventories that are prioritised
by value;
Ensuring a clear process to support a reduction in collections of „unnecessary data‟,
which maximises opportunities to streamline the volume of data we collect, and
ensures resources are focused on collecting essential data; and
Developing data.gov.uk and identify other digital channels to support users in finding
and accessing relevant high quality data and easy to use tools and applications?
Questions for consultation
The Government would welcome views on the following:
1. How should public services make use of data inventories? What is the optimal way to
develop and operate this?
2. How should data be prioritised for inclusion in an inventory? How is value to be
established?
3. In what areas would you expect government to collect and publish data routinely?
4. What data is collected „unnecessarily‟? How should these datasets be identified?
Should collection be stopped?
31
5. Should the data that government releases always be of high quality? How do we
define quality? To what extent should public service providers „polish‟ the data they
publish, if at all?
32
Government sets the example
Public service providers should lead the way in Open Data, ensuring that internal workings and the
underlying data behind advice and decisions are published
8.16 Repeated reports have found that failure to use
information effectively in government, whether as
management information or policy delivery data, leads to
ineffectiveness, higher costs, lower productivity, poor
26
outcomes and duplication. A failure to use datasets
intelligently in the provision of public services can mean
that we do not have a full understanding of the quality of
customer experience, for example an understanding of the
patient pathway from GP through hospital to outpatient.
At the same time, high volumes of FoIA queries to central
27
and local government require high levels of administrative
resource. If more data is made proactively available,
administrative burdens involved in answering requests may
be reduced.
8.17 Existing approaches like that of the London Borough
of Redbridge, which is actively promoting Open Data as a
way of managing down the number of FoIA and EIR queries
it receives, can be more widely applied. The key policy
challenge question is: how should government set an
example in its approach to Open Data. Should this be
through:
“Three basic principles
govern my work. First, data
is a public good and
therefore should be out
there. Second that if it is
available by Freedom of
Information then citizens or
residents shouldn’t have to
go through any bureaucratic
nightmares to get it, and
[third] that we should have a
presumption of openness –
extremely important in
restoring public confidence in
public institutions…”
Director of Digital Projects,
Greater London Authority
Routinely publishing evidence and databases behind policy statements in the way that
it does on Budget statements;
Routine publication of the data underlying surveys at the same time as the survey
analysis is published; and
Examining ways for improving the use of existing published data for policy and
research purposes?
Questions for consultation
The Government would welcome views on the following:
1. How should government approach the release of existing data for policy and
research purposes: should this be held in a central portal or held on departmental
portals?
26 NAO, A Short Guide to Structured Cost Reduction 2010
27 Figures from MOJ and UCL Constitution Unit
33
2. What factors should inform prioritisation of datasets for publication, at national, local
or sector level?
3. Which is more important: for government to prioritise publishing a broader set of
data, or existing data at a more detailed level?
34
Innovation with Open Data
8.18 The key policy question is to what extent government will need to support and
stimulate the development of an effective information marketplace in order to deliver the
benefits of Open Data. There is emerging evidence
which suggests innovative use of Open Data will, in the
long term, create the demand needed to embed the
“If people put data on the
broader culture change that we have described in the
web – government data,
public sector, in addition to stimulating the economy,
scientific data, community
offering new services and products that will transform
data, whatever it is – it will
the public and private sector, and creating jobs and
be used by other people to
opportunity for individuals.
do wonderful things in ways
8.19 A cost effective way forward may well be to
stimulate and challenge innovation based on data reuse, through setting up competitive challenges. There
are already good examples of this, including
Challenge.gov and New York City Challenge in the US
28
and Open Data Challenge in Europe.
they would never have
imagined. The cry of ‘raw
data now’ has spread around
the world.”
Sir Tim Berners-Lee OM,
KBE, FRS, FREng, FRSA
8.20 Key to this approach is that public service
providers develop new collaborative ways of working
with data users, including commercial users – and where necessary actively engage in the
market to stimulate demand for data. The premise for growth is that public service
providers do not charge users for their data, but that they enable users to gain access to it,
so that they can use it as a platform for innovation or enterprise.
8.21 In the international context, the UK is keen to ensure that international and EU
institutions reflect similar values and will work with colleagues to ensure that happens. The
UK is already working to ensure that international aid from UK taxpayers reflects the
transparency agenda. Separately, as a sign of our ambition to be the most transparent
government in the world, we will build on the UK‟s international role in Open Data and
maximise the part it will play in the Open Government Partnership.
29
8.22 The following approach might help government discharge its role as a catalyst for
innovation in the use of Open Data:
Public service providers to report each year on how they are building collaborative
relationships with the user community, including the commercial sector, which
promote use of data;
Government to maximise UK role in the Open Government Partnership and
commission an International Open Data benchmarking initiative.
28 Challenge.gov; http://techcrunch.com/2011/06/21/nyc-bigapps-ideas/; http://opendatachallenge.org/
29 http://www.opengovpartnership.org/
35
Question for consultation
The Government would welcome views on the following:
1. Is there a role for government to stimulate innovation in the use of Open Data? If
so, what is the best way to achieve this?
36
9. Next Steps
9.1 This document sets out a range of proposals intended to support a step change in the
Transparency and Open Data agenda in the UK. These proposals are at an early stage of
development because we are keen to get input at a meaningful stage from service users,
enterprise, developers and others.
9.2 The consultation will commence on 4 August 2011 and will last for 12 weeks. The
deadline for contributions is 27 October 2011 following which we will review responses and
firm up proposals in a White Paper in the autumn with appropriate impact assessments.
37
10. How to respond during
consultation
Online: www.data.gov.uk/opendataconsultation
E-mail: opendataconsultation@cabinet-office.gsi.gov.uk
Postal: Send a written response to:
Open Data Consultation
Transparency Team
Efficiency and Reform Group, Cabinet Office
1 Horse Guards Road
London SW1A 2HQ
Complaints or comments on the Cabinet Office consultation
process
If you have a complaint or comments on the consultation process itself, please contact:
Karen West
Finance and Estates Management, Cabinet Office
Rosebery Court
Norwich
NR7 0HS
E-mail: karen.west@cabinet-office.gsi.gov.uk
Handling of information from individuals
The information you send may need to be passed to colleagues within Cabinet Office or
other government departments, and may be published in full or in a summary of responses.
All information in responses, including personal information, may be subject to publication
or disclosure in accordance with the access to information regimes (these are primarily the
Freedom of Information Act 2000, the Data Protection Act 1998 and the Environmental
38
Information Regulations 2004). If you want your response to remain confidential, you should
explain why confidentiality is necessary and your request will be acceded to only if it is
appropriate in the circumstances. An automatic confidentiality disclaimer generated by your
ICT system will not, of itself, be regarded as binding on the Department. Contributions to
the consultation will be anonymised if they are quoted. Individual contributions will not be
acknowledged unless specifically requested.
39
Annex 1 – Best Practice and
Emerging Evidence
Accountability
A1.1 The Transparency and Open Data agenda has, until now, focused largely on holding
politicians and public bodies better to account. Research by polling company Ipsos MORI
suggests an indirect benefit arising from public
information, namely that the more citizens feel
informed the more they tend to be satisfied with public
“Publicity is justly
services and their local authority. Ipsos MORI also
commended as a remedy for
shows that overall satisfaction with local authorities is
social and industrial
further heightened when an informed public feel they
diseases. Sunlight is said to
can influence local decision making.30 The Royal
be the best of disinfectants;
Borough of Windsor and Maidenhead is leading the
electric light the most
way in publishing linked Open Data that enables
efficient policeman.”
individuals, suppliers and buyers to compare, relate and
understand spending decisions quickly and easily.
Justice Louis Brandeis,
Transparency allows for a virtuous circle of
‘Other People's Money—
information, accountability and user satisfaction.
and How Bankers Use It’
(1914)
A1.2 Accountability of spending is also particularly
important since government and public services spend
money on behalf of citizens and service users. This
demands a higher standard of scrutiny about value for money, efficiency (minimisation of
waste) and productivity. As examples from healthcare (e.g. Swedish disease-based data
repositories) and local authorities (e.g. Royal Borough of Windsor and Maidenhead) show,
transparency creates the possibility for public scrutiny, enables learning from best practice
and drives up productivity as a result.
A1.3 The London Borough of Redbridge is another example of good practice, pushing its
commitment to transparency beyond statutory requirement. The results have been
promising for Redbridge - since launching its own data portal, „DataShare‟, the local authority
gets up to ten thousand visitors to the site each month. DataShare currently links to its 17
datasets, which range from information on council expenditure and senior staff salaries, to
local properties subject to business rates.
A1.4 Police.uk is an information service for England and Wales, which provides data about
crime and policing in localities. After entering their postcode, town, village or street name,
users have instant access to street-level crime maps and data, as well as details of their local
policing team and beat meetings. From the information provided, users can create a crime
30 http://www.ipsos.com/public-affairs/sites/www.ipsos.com.publicaffairs/files/documents/what_do_people_want_need_and_expect_from_public_services.pdf
40
map of their local area, including the location of incidents – demonstrating accountability of
public service delivery. Since its launch, police.uk has received 423 million hits, 3,388 pieces
of individual feedback have been posted to the site by the public, and of its users 62% feel
better informed about crime and antisocial behaviour. The data on the site has also been
used to build iPhone, Android and Windows Phone applications.31
32
A1.5 Originally built almost entirely by a small group of volunteers, TheyWorkForYou is
a prime example of the type of useful app that can engage citizens in holding local authorities
to account. The site aggregates data available from Hansard, the Register of Members'
Interests, election results and voting records. The site has become a huge success and in
2008 The Daily Telegraph rated it 41st in a list of the 101 most useful websites.
33
A1.6 Public information and transparency are part of what the Transparency
Accountability Initiative calls „social accountability‟. It defines these mechanisms as including
„complaints mechanisms, public information/transparency campaigns, citizen report cards and
score cards, community monitoring and social audits‟. Together with the kind of public
expenditure monitoring described earlier, a new form of „bottom-up‟ regulation is becoming
more prevalent.
A1.7 Unfortunately the power of top-down regulatory bodies to spot and prevent adverse
events is frequently proven insufficient. This is much to do with the fact that risks in the
delivery of public services can never be eliminated. However, initial evidence (e.g. patient
safety in elderly care homes) suggests that Open Data and transparency can complement top
down regulatory structures, helping to reduce systemic risks and mitigate the impact of
failures.
A1.8 For example, data on deaths amongst the elderly from preventable causes revealed a
doubling of the number dying from dehydration between 1997 and 2010, and a sevenfold
increase in deaths from so-called „superbugs‟. In light of this and other information
presented to government, an inquiry has since been launched into the standards of care
delivered by NHS-funded care homes.34 The government is also now reviewing its
inspection and regulatory regime, again proving data can be a powerful tool for
accountability and improved outcomes.
A1.9 In 2009, the Care Quality Commission (CQC) moved pro-actively to investigate a
hospital in Stafford because evidence in the public domain revealed its mortality outcomes
were significantly higher than expected. This is itself an example of transparency of
information supporting better regulation of services. Both the subsequent Mid-Staffordshire
inquiries concluded that transparency had been the key driver for CQC investigation and
had saved lives as a result. But both inquiries emphasised that had greater use of data sharing
31 http://www.police.uk/
32 www.theyworkforyou.com
33 http://www.telegraph.co.uk/technology/3356874/The-101-most-useful-websites.html
34 http://byline.timetric.com/2011/02/01/deaths-from-malnutrition-the-missing-ons-data/
41
and user feedback been made by the hospital, the extent of negligence would not have
persisted for so long.
35
Choice
A1.10 Accountability is an important benefit, but is by no means the only benefit of Open
Data and Transparency. Comparative information enables choice to become a reality in
36
public services, something that evidence suggests matters to citizens. While many of the
public do not associate choice with an ability to drive up
quality standards, the evidence shows that choice can be
an effective mechanism for improving standards.
37
A1.11 If choice is to have meaning, it must be based on
intelligible information in the public domain. As
consumer experience in the UK energy market over the
last ten years has shown, empowered with information,
we can make more informed choices about the services
we use. Since the UK markets were opened to
competition in the late 1990s, a significant number of
consumers have switched energy supplier to get the best
tariff for their needs.
“When asked what help they
might need with a personal
budgets… information and
advice were the close first
and second priorities for all
groups…more than half (57
per cent) of council funded
care users said they would
need information on what to
spend their personal budget
on.”
‘Personal Best’ (2010),
A1.12 Comparative information websites have
Demos
transformed consumers‟ ability to search the energy
market in their own time, without supplier pressure, and
compare and contrast the tariffs and deals of particular
suppliers. Over 100,000 people now switch energy supplier or tariff each week, and in some
circumstances this can save consumers up to £200 per year.38 According to an Ofgem
survey, four-fifths of consumers point to price as the main reason for switching. The
availability of pricing data through price comparison websites is a huge driver behind this
behaviour.39
A1.13 Open Data can also be seen supporting choice in the Food Safety sector. In 1998 Los
Angeles County, USA introduced hygiene quality grade cards for display in restaurant
windows. Research from Stanford University shows that these grade cards caused
consumers to become more sensitive to changes in restaurant hygiene quality. It also
resulted in the number of people admitted into hospital with food-borne illness decrease and
the health inspection scores to increase. These improvements are thought to arise from
35 http://www.midstaffspublicinquiry.com/home
36 http://www.ipsosmori.com/DownloadPublication/1345_sri_what_do_people_want_need_and_expect_from_public_services_
110310.pdf
37 http://clients.squareeye.net/uploads/2020/documents/ESRC_Allan%20and%20Burgess_FINAL.pdf
38 http://www.consumerfocus.org.uk/news/confidence-code-can-help-consumers-get-a-better-energy-deal
39 http://www.ofgem.gov.uk/Markets/RetMkts/ensuppro/Documents1/Energy%20Supply%20Probe%20%20Initial%20Findings%20Report.pdf
42
consumers switching away from low hygiene restaurants. The competitive effect of
publishing scorecards served to ratchet up overall quality.
40
A1.14 The Car Theft Index was built to show the rate at which different types of vehicles
are stolen, informing customers. The Index was compiled from data taken from the Police
National Computer and information provided by the Driver and Vehicle Licensing Agency on
the number of cars on the road. The Index and its star rating system helped to make vehicle
security a marketing issue for manufacturers, as well as providing valuable information to the
insurance industry. In addition, while individual consumers were able to make a more
informed choice in their car purchases, the social gain has been seen in the fall of recorded
offences against vehicles by around a half (54%) since 2002/03.
41
A1.15 Increasingly, online tools are becoming available to inform choice in the public sector.
In the USA, there are at least three major websites allowing comparison of higher education
colleges and universities (e.g. CollegeMeasures.org, Federal Education Budget Project and
the US Department of Education). The primary motives for this are to facilitate better
accountability, improve choice and drive up outcomes. In the UK, Open Data has been used
to develop School-o-Scope, which presents data published by Ofsted and the Department of
Education in an easy-to-digest way so that users can access relevant and meaningful
information on how their local schools are performing.
42
A1.16 Seminal research in the early 1990s showed the impact of public reporting of
mortality rates in New York; those physicians and hospitals publishing better health
43
outcomes subsequently saw their market share grow. In the UK, NHS Barnsley has shown
the effect of simple kite marking information on choice. Fourteen GP practices, serving 40%
of the local population, were accredited with Barnsley‟s own „Green Tick‟ professional
standards kite mark. Between the launch of the scheme in 2008 and April 2011, 4,500
patients have so far chosen to switch to one „Green Tick‟ practices. On finding their GP
practice had been validated, one patient said: "I have been a patient [here] for many years but
seeing that the practice has received the award assures me that I am in the right place to receive
the care I need when I need it."
A1.17 Ready access to individual records held by government and public services is also
vital to make choice real. In healthcare, for example, choice of provider is illusory if any
decision other than your current GP and local hospital means reproducing your full medical
history from scratch. This will require patients to have access to information about
providers and their own record. Some practices have made online access to health records
available to their patients but few practices have activated or publicised this capability. Direct
40 http://www.stanford.edu/~pleslie/restaurants.pdf
41 http://www.homeoffice.gov.uk/publications/science-research-statistics/research-statistics/crimeresearch/hosb1210/hosb1210?view=Binary
42 http://www.guardian.co.uk/news/datablog/2011/jan/21/data-gov-nigel-shadbolt-government
43http://www.health.org.uk/public/cms/75/76/313/554/Public%20release%20of%20performance%20result.pdf?re
alName=UWXIXp.pdf
43
patient access to personal records will also drive data quality since patients will be able to
check the quality of data held about them.
A1.18 Around 55-60% of GP practices in the UK work with the same software vendor,
EMIS which, in collaboration with PAERS, has developed a platform for providing patients
with access to their own electronic GP records, securely, for free. Patients access their
records directly through the PAERS patient portal. PAERS is a UK company that works
closely with patients and healthcare professionals to provide patients with safe and secure
access to their full medical record, both online and via kiosks. It gives patients the
opportunity to see and understand their test results, to understand and to correct their
medical records. This enables them to make more informed choices between treatment
options, receive safer and more integrated care, save themselves and their practice time and
enjoy better relationships with their clinicians. PAERS also creates the capacity for patients
to share their records with a range of possible health and social care organisations anywhere
in the world, making choice real.
A1.19 Finally, the opportunity for user feedback presented by Open Data creates huge
potential for driving choice and subsequent service improvement in public services. Online
information and search tools have transformed the way we book flights and plan our
holidays. Sites like Trip Advisor contain hundreds of thousands of reviews from their travel
community, helping others to decide where – and where not – to visit. Similarly, Patient
Opinion, an online feedback platform for health services, shows how feedback posted by
patients and carers can be directed not just to the providers of care, but also to
commissioners, regulators, civil society organisations and others. By creating structured
public conversations about recent experiences of a local health service, Patient Opinion aims
to both stimulate improvement and show transparently whether services are listening to
those they serve. Examples of transparency in action include:
Prescribing practice in addiction services across a large Partnership Trust was
changed after a service user pointed out that the practice of ending prescriptions on
Fridays led to an increased risk of re-offending over the weekend.44
NHS Devon's new telephone referral service (DART) has received continuous online
feedback from patients since it launched in summer 2010, and has to date logged 11
improvements to both technical and administrative systems as a result.45
In Hull, clinic appointment systems were changed and information for patients was
improved following online patient feedback that waiting times at the sexual health
clinic were sometimes up to three hours.46
Similarly, a primary care addiction service in Sheffield installed an electronic waiting
time display after receiving repeated online feedback from users that late running
44 http://www.patientopinion.org.uk/opinions/41813
45 http://www.patientopinion.org.uk/services/5QQ-dart
46 http://www.patientopinion.org.uk/opinions/39759
44
clinics were causing stress and hostility between users and staff. Once waiting time
information was routinely available, relationships improved significantly.47
Productivity
A1.20 In 1999 the city of Baltimore elected a new mayor, Martin O‟Malley. Determined to
break the rampant culture of absenteeism amongst government officials, O‟Malley
implemented a data-tracking and management tool called CitiStat. CitiStat enabled the
mayor‟s office to monitor overtime and sick leave in real time, and resulted in a saving of
$13.2 million in its first year alone. Excluding the police
department, overtime fell by 40% in the first three years
and some agencies saw their level of absenteeism fall by as
“Currently, the reaction of a
much as 50%. By 2007 the city had made $350 million in
city can take a long time
efficiency savings – all whilst using affordable, off-the-shelf
because of a lack of clear
software and never more than eight full-time staff to
information…there is no
manage the CitiStat programme. In total, the programme
organized way to understand
cost Baltimore $285,000 to set up and requires
how occurrences in a city
approximately $400,000 per year to run.
relate to one another
other…[Real time analysis of
A1.21 CitiStat‟s focus is on data collected and used
data] gives the city
internally by government, demonstrating only part of the
management a more holistic
benefit of Open Data – bringing the data together such that
view…and the opportunity to
it can be compared and using the data to scrutinise
better understand the effect
performance. Productivity is driven by comparable data
of any decisions.”
being available to providers and commissioners to track
IBM
performance and respond accordingly, even if the public
don‟t engage with the data. Yet greater transparency of
outcomes and outputs (by putting this data in the public
sphere) further catalyse productivity improvements. Studies
have shown that even the possibility of scrutiny by peers and the public is enough to
incentivise behaviour change. In particular, work by social psychologists and behavioural
economists‟ shows people are sensitive to their self-image within peer groups and the
possibility of wider public scrutiny increases pro-social behaviour all the more.
48
A1.22 Use of Open Data to increase productivity in the US can be also seen in the
Veterans‟ Health Administration (VHA), where rigorous data collection and monitoring is
used to identify and resolve pinch points in the system. Together with electronic medical
records, high quality data has helped VHA to reduce length-of-stay by a third and outpatient
visits by 20%; improve prescription accuracy to 99.997% (compared to a 92-97% national
average range); almost eliminate costly and unnecessary lab tests, and reduce average cost
47 http://www.patientopinion.org.uk/opinions/20552
48 For example, Ariely et al. (2009) found charity donations increased if they were publicized. Burnham and
Hare (2007) conducted a public good experiment and found that when subjects were watched by a robot
with eyes voluntary contributions significantly increased. See:
http://gupea.ub.gu.se/bitstream/2077/22808/1/gupea_2077_22808_1.pdf.
45
per patient by 25% to $5,000 (compared to $6,300 national average). By 2006, 80% of VHA
patients reported being more satisfied with their care.
49
A1.23 A further international example of transparency driving productivity improvements in
healthcare comes from Canada. In order to eradicate variable approaches to patient care,
several leading hospitals began using accurate, real-time data to monitor how closely their
doctors were following best practice. Focusing on the most common conditions, the
doctors‟ performance was measured and compared. In one hospital, researchers found
making this type of information available publicly to doctors led to rapid improvements in
50
performance and outcomes. This is because:
Publicly sharing performance and outcome data encouraged the communication and
dissemination of best practice;
Doctors whose performance was furthest from best practice tended to migrate to
the average fairly quickly; and
Top performers tended to improve as well (as they wanted to maintain their status
as top performers).
A1.24 In a few months the average length of stay decreased by more than 30% and
unexpected readmissions decreased by more than 20%. Effective use of data and
information can help public services to allocate their resources most effectively, reducing
waste and enabling more to be done with the same or less. Data-driven decision making
(especially in real time) is a key driver of productivity, helping to support continuous quality
improvement and cost-effective resource allocation. As well as limiting the costs associated
with managing Freedom of Information requests, transparency highlights variation in
procurement costs within and across public service providers, enabling government to
identify cashable savings.
A1.25 Transparency of data on inputs, outputs and outcomes in healthcare has revealed
levels of variation in quality and cost in health care systems across Europe. The impact of
public reporting of named institutions has been particularly effective amongst poorest
performers. Royal Bournemouth and Christchurch Hospitals Foundation Trust learned,
following analysis that its length of stay in orthopaedics was, at eight days, well above the
national average even after adjustment for other factors. This insight was used to inform a
service redesign programme that halved the number of bed days and saved £1 million.
51
A1.26 Open Data and transparency were also instrumental in reducing costs of employment
services in Germany and restoring public confidence in the relevant agencies. Lack of
comprehensive data on customer histories, costs and labour market characteristics had
hitherto prevented sufficient understanding of the impact of services and their value for
money. Lack of data also prevented clear communication with the public on the nature and
49 http://www.cbo.gov/ftpdocs/88xx/doc8892/MainText.3.1.shtml
50 http://www.mckinsey.it/storage/first/uploadfile/attach/140184/file/hi08_3clinicaloutcomes_final.pdf
51 http://drfosterintelligence.co.uk/wp-content/uploads/2011/06/RTM-overview.pdf
46
scale of the problem of unemployment. The challenge was to integrate 11 datasets of
different structure, format and data quality into one. From this, key performance indicators
were then developed so that outcomes data could be compared across regions. Regular
publication of data revealed significant variation and this transparency led to considerable
improvements in cost effectiveness and service efficacy. Over three years cost savings
amounting to €10 million were extracted from the system and a wealth of granular linked
data on German labour market performance was made available to academics, journalists
and other interested parties.
A1.27 Open Data initiatives are also being used to generate monetary savings from reduced
fuel consumption. As part of a series of integrated technological, environmental and
economic initiatives, the city of Amsterdam is working with the region‟s ICT and data
community to help drive up energy efficiency. It is hoped that effective use of ICT can
contribute a 20% reduction in the city‟s carbon emissions by 2025 – half of its 40% target
(based on 1990 levels). Green ICT Amsterdam Region project has been a year in operation
and has already secured €15 million in private–public partnership funding for its first phase of
pilots. Collaborative use of data, smart metering and public scrutiny of progress is at the
heart of Amsterdam‟s Green ICT project.52
A1.28 An online benchmarking tool for the oil industry brought together over 200 metrics
from 130 datasets across all oil producing regions. Visibility of comparative performance and
cost data served to highlight variation in the industry and identify best practice. The
resulting diffusion of best practice and knowledge of relative cost/outcomes data resulted in
average cost improvements of 10% across the industry, worth $100 million over 10 years.
53
A1.29 Where consumers of services can be encouraged, sharing personal data collected by
a public body with the user can result in improvements in outcomes. A power company in
the US sent personalised bills to 35,000 randomly selected customers, rating them on their
energy use compared with that of neighbours in 100 homes of similar size that used the
same heating fuel.54 The customers were also compared with the 20 neighbours with the
best fuel consumption and after six months it found that customers who received the
personalized report reduced energy use by two percent more than those who got standard
statements. The approach has now been adopted across that company and is being trialled
or rolled out at more than a dozen power companies across the US, to hundreds of
thousands of customers.
52 http://www.accenture.com/SiteCollectionDocuments/PDF/Accenture_2011_GoTFC_ResearchDriving_Public_Entrepreneurship.pdf
53 McKinsey & Co. analysis. For more information, see e.g.
http://www.pilottaskforce.com/docs/aboutpilot/atemplateforchange.pdf.
54 http://www.nytimes.com/2009/01/31/science/earth/31compete.html
47
Quality and Outcomes
A1.30 International evidence suggests transparency is a powerful way of improving quality in
public services. More informed choice and public sector decision making can be drivers of
this improvement in outcomes. The opportunities to identify best practice as well as the
behavioural effects of public scrutiny can also be a driver of quality. For example, North
West London Hospitals NHS Trust focused on improving its patient safety record by
utilising internal and accessible data. Within a year, this
resulted in 174 fewer deaths amongst eight treatment
areas targeted and 255 fewer deaths in all diagnoses.
The hospital‟s standardised mortality ratio fell, becoming
“It’s estimated
the lowest of the English acute trusts in 2007-08.
55
A1.31 Another powerful UK example comes from the
Society of Cardiothoracic Surgeons which reported in
2010 that mortality in coronary artery surgery had fallen
by a fifth and in aortic valve replacement surgery by a
one third over five years. This result was attributed by
the professional body to public reporting of outcomes.
Contrary to oft-quoted concerns, the Society also noted
that there was no evidence that public reporting had led
to „gaming‟ by surgeons by taking on less risky cases and
net savings outweighed the costs of data collection
56
more than threefold.
that[individual clinical
outcome data has] probably
saved up to a thousand lives
a year, and it’s done that by
resulting in much more
reflective practice among the
heart surgeons in this
country...
Our heart surgery is now
measureable, demonstrably,
and statistically better than
anywhere else in Europe…”
NHS Medical Director
57
A1.32 A variety of organisations including CHKS , Dr
58
Foster and publicly funded Health Observatories59
have made data on mortality rates available across the
spectrum of in-patient activity in the NHS with significant and measurable impact on
reducing variation and absolute levels of avoidable death.
A1.33 Quality of Swedish secondary care was shown to improve after individual hospital
performance data was published. Prior to 2006 hospitals could not be identified so data on
quality revealed trends, not performance by named provider. After 2006 data on individual
hospital performance was published, attracting significant attention from the media and the
public. Overall quality rose and the overall degree of variation between all hospitals
narrowed, with improved adherence to best practice clinical guidelines seen nationally. The
55 http://www.bmj.com/content/340/bmj.c1234.full?sid=766e59be-ff67-4ab1-974e-034286f9072e
56 http://www.guardian.co.uk/lifeandstyle/2009/jul/30/heart-surgery-death-rates-fall
57 http://www.chks.co.uk/
58 http://drfosterintelligence.co.uk/
59 http://www.apho.org.uk/
48
greatest effect was upon the poorest performers, where transparency significantly
accelerated both the magnitude and the rate of clinical improvement.
60
A1.34 There is also evidence demonstrating the effect of transparency on driving up quality
in education. Until devolution in 1999 England and Wales shared a common education
system and outcomes in the two countries, as monitored by National Statistics, diverged
little between 1998 and 2001. After 2001, school league tables were abolished in Wales, but
in England the ranked publication of GCSE results by school continued. Comparing the
results of cohorts before (1992-2001) and after the policy change (2002-2008) shows Wales
experienced an average drop of 3.4 percentage points in the proportion of pupils in each
school achieving at least five good GCSE passes.61 This stemmed from a number of factors,
but analysis by the London School of Economics and Bristol University suggested that the
divergence was at least in part due to the impact of transparency.
62
Social Growth
A1.35 Open Data has already shown itself to be a facilitator of „social growth‟. By social
growth, we refer to three main types of increased engagement:
Individual self-service (e.g. providing information and appropriate channels for patients
and carers to take greater responsibility for their own health);
Citizen participation (e.g. encouraging
individuals and community to engage in more
informed debate about local budgeting or
planning proposals); and
Citizen collaboration (e.g. working together to
set up a Free School or another form of
public service mutual).
A1.36 Giving people access to their data and
changing the way in which services do business
presents opportunities for public service
transformation by giving users more power to selfserve. In 2000 Humana, an American healthcare
provider, started sending its customers individual
information about their benefits, claims, costs of
care, and risk profile. By individuals taking greater
responsibility for managing their own health and by
realising their actual treatment costs and health risks,
Humana saw a decrease in admissions of 30%.
“[With new
technologies]...nowadays
even a relatively small
business can go from local to
national and then global
operation in a short amount
of time. Fortunes can be
made by providing goods
and services at a low price to
a global market of 6 billion
people.”
Chief Economist, Google
60 http://www.bcg.com/expertise_impact/publications/PublicationDetails.aspx?id=tcm:12-64828
61 http://www.bristol.ac.uk/cmpo/publications/papers/2010/wp246.pdf
62 http://content.yudu.com/A1slnt/BetterUKSpr11/resources/index.htm?referrerUrl=
49
A1.37 In the UK NHS Choices provides users with access to expert online medical
information. It is the country's biggest health website and provides users with the necessary
information for them to make informed choices about their health. Research has shown the
availability of NHS Choices has modified demand for more resource intensive face-to-face
medical advice. Conservative estimates suggest NHS Choices saves approximately £44
million per year by freeing up healthcare capacity.63
A1.38 The same result can be seen in education where online access to assignments and
school records has improved attendance and reduced the dropout rate amongst students at
Mary‟s County Public Schools in the United States.64 Up-to-the minute attendance
information has encouraged parents to become more involved in the education of their
children supporting an improvement in attendance and attainment.
A1.39 Another example of increased individual self –service comes from the US Veteran‟s
Association, which introduced a Health Buddy System allowing patients to take a more active
role in monitoring their health and symptoms whilst at home. Data from the Department of
Veterans‟ Affairs demonstrated a 19% reduction in hospitalisations and 25% reduction in
65
„bed days of care‟ for participating patients using the system. In the UK there is evidence
that patients tend to opt for less invasive, less costly treatments when they are fully
informed and encouraged to participate in treatment decisions. Trials suggest surgical rates
could reduce by up to 25% if patients were actively involved in this way, leading to significant
cost savings.
66
A1.40 Open Data and transparency also create a platform for more informed public debate
and citizen engagement. For example, OpenlyLocal67 in the UK and Miami 311 in the US
shows how Open Data can support citizens to engage with their local public services and
government, enabling more responsive, efficient and effective services. By reporting nonemergency issues via telephone and then tracking the status of the city‟s response online,
Miami‟s mapping tool enables citizens to see where other problems have been reported in
their neighbourhood. Miami recently launched another mapping tool to inform the public of
new and ongoing capital improvement projects in the city.68
A1.41 FixMyStreet is another UK example, helping users to find the right telephone number
or form to report local problems, ranging from dog fouling to broken streetlights. Since its
launch, FixMyStreet has received over 90,000 citizen reports. In June 2011, 2,086 reported
problems were fixed and 149,001 progress updates on reports were made.
63 http://www.nhs.uk/aboutNHSChoices/professionals/developments/Documents/annual-report/primary-careconsultation-report.pdf (p.2)
64 http://www.sungardps.com/~/media/Campaigns/Corporate/Education/CaseStudies/K12_StMarysCountyPublicSchoolDistrictMD.ashx
65 http://www.bosch-telehealth.com/content/language1/html/6854_ENU_XHTML.aspx
66 http://www.guardian.co.uk/commentisfree/2010/apr/09/patients-involved-nhs-treatment-decisions
67 www.openlylocal.org.uk
68 http://maps.miamigis.com/capital_improvement/
50
69
A1.42 Hyperlocal websites such as Kings Cross Environment in London or the Ventnor
70
Blog in the Isle of Wight are often established by groups of citizens focusing on the
71
concerns of small geographical areas, such as a neighbourhood. Other sites may be
collaborations between local authorities and web developers, for example Sutton Bookshare,
created by the London Borough of Sutton as a resource for community borrowing of books,
or 'Who Owns My Neighbourhood?‟ created by Kirklees Council in partnership with
Thumbprint Co- operative using land ownership data. The app allows residents to add and
discuss information about their area, and keep up to date with solutions to local land
problems such as fly-tipping.72
A1.43 With access to more information the public is better equipped to hold local, and
central, government to account. Data on service productivity, costs and delivery outcomes
can also empower citizens and communities to engage in the Big Society: by creating an
online app, mash-up or tool using Open Data, or by establishing an employee-owned mutual,
or a parent/teacher-led Free School.
A1.44 A group of teachers who set up a Free School in Bedford grounded their decision
upon knowledge of relatively poor outcomes in the area. Of the 70% of students who
attend the local senior school, ―only 31% managed to get five good GCSEs including English and
maths. Other schools in our town are doing better – although in Bedford itself, only 42% of 16-yearolds this summer achieved the same measure.‖
needed as a result.
73
The group believes that a “fresh approach‖ is
A1.45 Another example of how Open Data can encourage collaboration between citizens is
74
PatientsLikeMe. This website enables people with similar healthcare needs or concerns to
learn from each other by sharing their experiences online. Over 13,000 patients have public
profiles that allow others to search for people with who they might relate (e.g. whether by
age, condition or treatment). The free online member community provides information
from user-generated content as well as research undertaken with patients, academics,
industry and other healthcare partners.
69 http://www.kingscrossenvironment.com/
70 http://ventnorblog.com/
71 http://www.gohyperlocal.com/
72 http://www.nesta.org.uk/areas_of_work/public_services_lab/make_it_local
73 http://www.guardian.co.uk/commentisfree/2010/nov/20/free-school-education-extra-freedoms
74 http://www.patientslikeme.com/
51
Economic Growth
A1.46 Finally, Open Data has the potential to drive
economic growth. A small market place has already
developed in the UK in the health sector. Companies
such as Dr Foster and CHKS have formed the basis of a
fledgling industry with an estimated total value of
around £50 million per annum. However, they are
currently limited by their ability to access, analyse and
publish anonymised data. The NHS Information Centre
has estimated that the true scale of the market in
hospital data alone is £250 million; far more if other
NHS data were made available. The information market
place in financial services globally is now estimated to be
75
worth $23.2 billion. The potential for a health and
social care information market is no less ambitious.
A1.47 Together with linked Open Data, cloud
computing and the potential of the Semantic Web, the
Royal Society anticipates that new forms of scientific
analysis will arise. Four members of the Royal Society‟s
Working Group on Open Data explained just one of
the potential areas for enterprise and public value from
Open Data:
“Because we collect
hundreds of thousands of
ECGs per year, we can
analyze heart rhythms by
gender, race, and age.
Because we create
thousands of lung images
per year, we can create lung
models for computer-aided
diagnosis.
Because we have a database
of over one million ICU
patients – their monitoring
and lab data – we can create
sepsis algorithms to predict
and treat.”
CEO Health Informatics,
Phillips
―The meta-analysis of the raw data from clinical trials of the effects of aspirin in the prevention of
cardiovascular disease including—as it did—data from 95 000 patients is a fine example of the
benefits of data-sharing. With the increasing use of electronic medical records, there is the
opportunity for anonymised data from routine clinical use of drugs to provide high quality
pharmacovigilance on a hitherto unprecedented scale.‖
76
A1.48 There are similar opportunities in the education sector. The UK already has the
77
largest e-learning market in Europe, worth an estimated £472 million in 2010. By creating
the potential for expansion of services into educational data analysis, the size of the market
itself is likely to expand. An education publisher, Pearson, is enabling teachers and schools
to access examination results. For example, teachers can see how close students were to
grade boundaries and help students decide whether to resit an examination; identify skills
75 IRN research report 'The Global Financial Data market' August 2010
76 Geoffrey Boulton, Michael Rawlins, Patrick Vallance, Mark Walport, „Science as a public enterprise: the case
for Open Data‟ The Lancet, Volume 377, Issue 9778, Pages 1633 - 1635, 14 May 2011
77 http://www.e-learningcentre.co.uk/Resource/CMS/Assets/5c10130e-6a9f-102c-a0be003005bbceb4/Learning%20Light%20-%20The%20UK%20e-learning%20Market%202010%20%20Summary%20v2.pdf
52
gaps through results question by question, and devise suitable learning and revision plans for
78
the following academic year.
A1.49 Open Data also has a key role to play in driving new business models and applications
for geospatial information. In 2010 the Association for Geographic Information (AGI)
estimated that the level of investment in the UK geographic information market, both in the
79
public and private sector, is between £650 million and £900 million per annum. The AGI
anticipates that the market will grow to over £1 billion by 2015. In particular, it expects
growth to come from: homeland security, climate change, disaster management, energy and
food security.
A1.50 Another driver will be „smart logistics‟ stemming from environmental concerns, the
high cost of fuel, and an increase in home delivery of retail and groceries. Traditionally
geospatial applications have been used to document or analyse the past or plan for the
future. It is likely there will be significant growth in applications focused on real time
information; already over 15% of applications on Apple‟s AppStore utilise location in some
way.80 One German study suggested the value of spatial data is such that, on top of the
economic value of basic public sector information “calculations for value added and
employment can be higher by approximately a factor of two if all data are provided localised with a
81
spatial reference in the future.” The AGI highlights the speed of growth of consumer
applications based on geospatial information; their expectation is that the Location Based services
market is likely to more than triple from the £50 million estimated currently.
A1.51 Data collected by the US National Weather Service supports a huge industry.
According to the American Meteorological Society, the total size of the private sector
weather market is greater than $1.5 billion per year82 while research has estimated the
direct economic value of access to US government meteorological data is $500 million per
83
year . A literature review by Arzberger et al (2004) also identified the role of this public
data in supporting a rapidly growing weather risk management industry underwriting financial
risk management instruments, valued at approximately $8 billion. In contrast, the same
study concluded: “the private-sector value adding industry for meteorological information in the
European Union is very small, largely attributable to the highly restrictive data policies of most
84
national governmental meteorological services.”
A1.52 The UK is an exemplar of open government in Europe and the rest of the world, but
still more could be done. The US health informatics market indicates the potential market
78 https://www.edexcel.com/resultsplus/Documents/rp_guide.pdf
79 http://www.nao.org.uk/publications/1012/defra_gis.aspx
80 http://www.agi.org.uk/storage/AGI%20Foresight%20Study%20Summary%20Report%201.1.pdf
81 Fornefeld, M., Oefinger, P., and Rausch, U., „The Market for Geospatial Information: Potentials for
Employment, Innovation and Value Added‟ (Bundesministerium für Wirtscaft und Arbeit, 2003)
82 http://www.ametsoc.org/boardpges/cwce/
83 Weiss, P. (2003) Conflicting International Public Sector Information Policies and their Effects on the Public
In Esanu, J. M. & Uhlir, P. F. (Eds.), The Role of Scientific and Technical Data in the Public Domain:
Proceedings of a Symposium, Washington, D.C.: National Academies Press, 29-132.
84 http://www.jstage.jst.go.jp/article/dsj/3/0/135/_pdf
53
opportunities for economic growth that could arise from opening up data. At over 17% of
GDP, the US spends more than any other country in the world on healthcare. Efficiency is a
major issue, and ongoing discussions are closely linked with the introduction of healthcare
ICT systems and processes designed to increase automation of transactions, enable greater
self-care and make the diversity of proprietary systems interoperable. For example,
industries supporting billing automation, e-prescribing and online records are growing
rapidly.
A1.53 The inclusion of electronic prescribing in the Medicare Modernization Act (MMA) in
2003 helped to develop the industry now estimated to be worth $204 million and rising.
The number of electronic prescriptions increased 181% between 2008 and 2009 and the
85
transfer of prescription histories online increased fivefold over the same period. The
Health Information Technology for Economic and Clinical Health (HICTECH) Act (2009) has
also promoted rapid growth of the online medical records industry. For example, one of the
country‟s leading contractors – Epic.com – is expected to cover about 30% of the US
population once all its systems are in place. In 2009 its revenue already exceeded $650
million. Providers of healthcare informatics and ICT are large scale employers, with revenue
increasingly sourced from services as opposed to traditional forms of hardware or software.
Siemens Healthcare, for example, is the seventh biggest firm in the industry, employing
62,000 in the US alone and generating approximately $1.4 billion revenue (2009). Cerner,
McKesson and Philips Healthcare are other major players in the space. Combined their
global annual revenues exceed $5.5 billion (2009).
A1.54 The UK Government is already considering how to maximise the potential economic
gains from Open Data. In January this year, government announced its intention to create a
Public Data Corporation (PDC). This would bring together data-rich organisations with the
aims of:
Providing a more consistent approach towards access to and accessibility of Public
Sector Information, balancing the desire for more data free at the point of use, whilst
ensuring affordability and value for taxpayers;
Creating a centre of excellence driving further efficiencies in the public sector; and
Creating a vehicle that can attract private investment
A1.55 Government believes that a PDC will provide structures and incentives to promote
greater access to, and usage of, public data and information, delivering benefits and growth
for the wider economy. There are also significant opportunities to drive efficiency and
improvement of public services through better sharing of key data between
organisations. The Government is currently consulting on important questions on key
aspects of data policy for the PDC including charging, licensing, and regulation in preparation
for the constitution of the PDC in autumn 2011.
85 http://www.aafp.org/online/en/home/publications/news/news-now/practicemanagement/20100315surescriptsreport.html
54
A1.56 As part of the second phase of the Government‟s Growth Review there will also be a
specific work stream which will focus on, and bring more depth to, the economic benefits of
Open Data by assessing the size of the opportunities for government. This will more fully
elaborate the benefits, costs and tradeoffs needed in opening up data, and also look to
identify specific data sets through which to make immediate progress.
55
Annex 2 – Public Data Principles
Working definition of “Public Data”
"Public Data" is the objective, factual, non-personal data on which public services run and are
assessed, and on which policy decisions are based, or which is collected or generated in the
course of public service delivery.
Draft Public Data Principles
Public data policy and practice will be clearly driven by the public and businesses
who want and use the data, including what data is released when and in what
form – and in addition to the legal Right to Data itself this overriding principle should apply
to the implementation of all the other principles.
Public data will be published in reusable, machine-readable form – publication
alone is only part of transparency – the data needs to be reusable, and to make it reusable it
needs to be machine-readable. At the moment a lot of government information is locked
into PDFs or other unprocessable formats.
Public data will be released under the same open licence which enables free reuse, including commercial re-use – all data should be under the same easy to
understand licence. Data released under the Freedom of Information Act or the new Right
to Data should be automatically released under that licence.
Public data will be available and easy to find through a single easy to use online
access point (data.gov.uk) – the public sector has a myriad of different websites, and
search does not work well across them. It‟s important to have a well-known single point
where people can find the data.
Public data will be published using open standards, and following relevant
recommendations of the World Wide Web Consortium. Open, standardised
formats are essential. However to increase reusability and the ability to compare data it also
means openness and standardisation of the content as well as the format.
Public data underlying the Government‟s own websites will be published in
reusable form for others to use – anything published on government websites should be
available as data for others to re-use. Public bodies should not require people to come to
their websites to obtain information.
Public data will be timely and fine grained – Data will be released as quickly as possible
after its collection and in as fine a detail as is possible. Speed may mean that the first release
may have inaccuracies; more accurate versions will be released when available.
Release data quickly, and then re-publish it in linked data form – Linked data
standards allow the most powerful and easiest re-use of data. However most existing
internal public sector data is not in linked data form. Rather than delay any release of the
56
data, our recommendation is to release it „as is‟ as soon as possible, and then work to
convert it to a better format.
Public data will be freely available to use in any lawful way – raw public data should
be available without registration, although for API-based services a developer key may be
needed. Applications should be able to use the data in any lawful way without having to
inform or obtain the permission of the public body concerned.
Public bodies should actively encourage the re-use of their public data – in
addition to publishing the data itself, public bodies should provide information and support
to enable it to be re-used easily and effectively. The Government should also encourage and
assist those using public data to share knowledge and applications, and should work with
business to help grow new, innovative uses of data and to generate economic benefit.
Public bodies should maintain and publish inventories of their data holdings –
accurate and up-to-date records of data collected and held, including their format, accuracy
and availability.
Available at http://data.gov.uk/wiki/Public_Data_Principles
57