August 2011
Gemma Van Halderen
Statistical Data Integration
and Analysis Branch
Australian Bureau of Statistics
What’s all the excitement about?
• Information to address (solve?) those important issues of
today
• Homelessness, climate change, social inclusion, ageing
population
• Making better use of existing data
• ROI
• Filling data gaps
• Giving effect to legislation
• Maximising the use of data for official purposes
• A community of government officials demanding and creating
change
• National statistical service in practice
Journey so far
Leadership
Principles
Governance
and
Institutional
Arrangements
Leadership
•
Principles
Governance and
Institutional Arrangements
Joint proposal from Australian Statistician and
Secretary of Health and Ageing
•
•
•
to put in place a safe and effective environment for
integrating Commonwealth data for statistical and
research purposes
Portfolio Secretaries meeting (now Secretaries Board)
April 2009
•
Cross Portfolio Statistical Data Integration
Committee established
•
Jointly chaired by ABS and Department of Health
and Ageing
•
Represented by portfolio agencies
Leadership
•
Principles
Governance and
Institutional Arrangements
Principles for Statistical Data Integration endorsed
•
•
Portfolio Secretaries meeting (now Secretaries Board)
February 2010
•
Developed by the Cross Portfolio Statistical Data
Integration Committee
•
Represented by portfolio agencies
Commonwealth High Level Principles
Transparent
Governance
Treat data as a
strategic asset
Public Benefit
High
Level
Principles
Statistical and
Research
Purposes
Privacy and
Confidentiality
Custodian’s
retain
accountability
Integrating
Authority to
be nominated
Principle 3: Integrator’s Accountability
• An IA must be identified for each statistical data integration
proposal. This authority will be held responsible for the sound
conduct of the statistical data integration proposal, in line with
the agreed requirements of the responsible agencies
• Although the IA is the single organisation ultimately
accountable for the statistical data integration project, it may
work with a network of agencies to achieve the data
integration.
• The IA will ensure appropriate governance is in place including
•
•
•
•
•
•
An open approval process is followed
Documentation of the proposal
The impact on privacy
Risks have been assessed, managed and mitigated
The expected costs and benefits
The outputs
Principle 3 cont.
• A family of data integration projects using the same source
datasets, for similar purposes, with the same IA, may be
treated as a single program for the purposes of the approval
process
• The IA will be responsible for the ongoing management of the
integrated data, ensuring it is kept secure, confidential, and fit
for the purposes for which it was approved
• If it is an ongoing project, the IA will be responsible for
initiating and managing its regular review, in consultation with
source data agencies
Principle 3: Integrator’s Accountability –
ABS experience
• For integration projects involving Census data, ABS is the
nominated IA
• All data integration projects involving Census data are treated
as a family of projects
• ABS is ultimately accountable for Census data integration
projects
• An approval process was followed for the projects such as
Indigenous Mortality and Enhancing Cancer statistics
• Proposals have been documented and are available on the
ABS website
Principle 3: Integrator’s Accountability –
ABS experience
• Privacy management included a Privacy Impact Assessment (in
2006), consultations with Privacy Commissioners, and only
making use of name and address during Census processing
period.
• Risks have been assessed, managed and mitigated. For
example
• ABS is only developing a 5% Statistical Longitudinal Census
Dataset, not a 100% dataset.
• Datasets are deleted once purpose of the project has been met
• Close consultation with other data custodians e.g. Registrars
• Expected benefits are clearly outlined in the ABS Information
Paper
• Normal ABS governance processes are in place for outputs
Principle 2: Custodian’s Accountability
Each responsible agency for source data:
• Must agree mechanisms to achieve adequate control and
manage risk approach to their own situation. These
mechanism may include the use of particular institutional
arrangements with trusted institutions, the use of specified
standards and audits against those standards, and the
potential application of sanctions
• Will need to agree the nature of valid uses that can be made
of the integrated datasets and the approval mechanism to be
applied to applications to use the datasets, as well as any
control mechanisms to be applied to such use
Principle 2 cont.
Each responsible agency for source data:
• Will need to manage the potential increase in identifiability of data
for which they are responsible when it is used in conjunctions with
data from other sources. It will need to agree mechanisms by which
it can assure itself that outputs from the data integration are not
likely to enable the identification of individuals or businesses
• Will need to agree the final content of any new data integration
proposal, or any material change to an existing data integration
proposal as part of the approval process. They must be kept inform
of, and agree, more minor proposed changes to existing proposals.
When an agency does not agree to the use of its source data in a
statistical data integration proposal, the data will not be included.
Leadership
•
Principles
Governance and
Institutional Arrangements
Principles for Statistical Data Integration endorsed
•
•
Portfolio Secretaries meeting (now Secretaries Board)
February 2010
•
Agreement to establish governance and institutional
arrangements to give effect to the high level
principles
•
Developed by the Cross Portfolio Statistical Data
Integration Committee
•
Represented by portfolio agencies
Leadership
•
Principles
Governance and
Institutional Arrangements
Governance and Institutional Arrangements for
Statistical Data Integration endorsed
•
•
Secretaries Board
October 2010
•
Cross Portfolio Statistical Data Integration
Committee completed role
•
New governance arrangements involving portfolio
agencies established
Cross Portfolio Data
Integration Oversight
Board
Members: ABS, DHS, DoHA, FaHCSIA
Secretariat
Best Practice
Guidelines
Integrating
Authorities:
Accreditation
Process
Education
and Training
Network of
Survey
Liaison
Officers
Register of
data
integration
projects
Cross Portfolio Data
Integration Oversight
Board
Members: ABS, DHS, DoHA, FaHCSIA
Oversight Board
Terms of Reference
Role
• provide strategic and collaborative leadership, support effective
governance and help manage the risks of particular projects
• help manage the systemic risk associated with conducting
multiple data integration projects
• endorse changes or additions to the overall environment e.g. the
development of new general tools to support integration or safe
access to integrated data.
Cross Portfolio Data
Integration Oversight
Board
Members: ABS, DHS, DoHA, FaHCSIA
Oversight Board
Terms of Reference
Activities
• review data integration projects which pose a significant level of
systemic risk, advise on risk mitigation strategies, and review any
adverse incidents of high public concern.
• approve a comprehensive set of guidelines describing best
practice for data integration projects involving Commonwealth
data
• establish an accreditation process to enable the endorsement of
authorised and accredited Integrating Authorities with the
capacity to deal with high risk data integration projects.
Integrating
Authorities:
Accreditation
Process
HIGH RISK PROJECTS
1. Authorisation to receive identifiable data – by legislation or consent
2. Legal protections prohibiting the disclosure of identifiable data
MEDIUM AND LOW RISK PROJECTS
1. Authorisation to receive identifiable data – by legislation or consent
2. Policy environment which provides support to ensure that no identifiable
data is disclosed
Integrating
Authorities:
Accreditation
Process
Self-Assessment
IAs prepare report on
criteria for accreditation
Audit
Independent party audits
self-assessment
Decision
Oversight Board makes
final decision on
accreditation
Publication of list of accredited
agencies
Integrating
Authorities:
Accreditation
Process
Interim Accreditation Criteria
Ability to
ensure
secure data
management
IAs must
demonstrate that
information
that is likely
to enable
identification of
individuals
or organisations is
not
disclosed to
external
users
Availability
of
appropriate
skills
Appropriate
technical
capability
Lack of
conflict of
interest
Culture and
values that
ensure
protection
of
confidential
information
and support
the use of
data as a
strategic
resource
Transparency of
operation
Appropriate
governance
and administrative
framework
Interim Accreditation Criteria – ABS experience
Ability to ensure secure
data management
Adhere to separation
principle?
IAs must
demonstrate
that information
that is likely to
enable
identification of
individuals or
organisations is
not disclosed to
external users
Program of audits to
ensure continued
security of data?
Comply with Australian
Government Protective
Security Policy
Framework?
Have staff undergone
police checks prior to
employment?
Is access to agency’s
premises controlled?
Is the Internet gateway
secured? Regularly
reviewed by Defence
Signals Directorate?
Does the agency have
an Information Security
Policy and procedural
plan?
How is safe
access provided?
Availability of
appropriate
skills
What expertise
and experience
does the agency
have to
undertake high
risk data
integration
projects? What
strategies are in
place to acquire
the necessary
expertise?
What
documentation
and training is
available to
ensure staff
have the
appropriate
skills and
knowledge
required in high
risk data
integration
projects?
Appropriate
technical
capability
Lack of
conflict of
interest
Does the
organisation have
secure IT
infrastructure,
including
hardware and
software systems,
with the capacity
to support the
potentially large
and/or complex
files associated
with high risk
data integration
projects?
Does the
agency
have a
compliance
monitoring
or
regulatory
function?
Is the system
designed so that
access and
changes to data
can be audited by
date and user
identification?
Does the system
‘footprint’
inspection of
records and
provide an audit
trail?
What IT support is
in place for staff?
Culture and values
that ensure protection
of confidential
information and
support the use of
data as a strategic
resource
Are there mechanisms
in place to engage with
stakeholders to
maximise the
usefulness of the
statistics?
Transparency
of operation
Are data
revision
statements
published?
Are details of
governance
arrangement
s published?
How does your agency
demonstrate that it
provides for valuable
use of the data?
Are detail of
data
integration
projects
published?
Is an appropriate
culture and values
embedded in a
corporate plan/mission
statement/
policies etc?
What other
relevant
material is
published?
Have staff been
trained in
requirements for
protecting personal
information and are
they aware of policies
regarding breaches of
security or
confidentiality?
Appropriate
governance
and
administrative
framework
What are the
institutional
and projectspecific
governance
arrangements
for data
linking?
What
framework is
in place to
conduct
investigations
and handle
complaints?
Cross Portfolio Data
Integration Oversight
Board
Members: ABS, DHS, DoHA, FaHCSIA
Secretariat
Best Practice
Guidelines
Integrating
Authorities:
Accreditation
Process
Education
and Training
Network of
Survey
Liaison
Officers
Register of
data
integration
projects
Best Practice
Guidelines
High Level Principles
Transparent
Governance
Treat data as a
strategic asset
Public Benefit
High Level
Principles
Statistical and
Research
Purposes
Custodian’s
retain
accountability
Privacy and
Confidentiality
Confidentiality
Integrating
Authority to
be nominated
Principle 6: Preserving Privacy
and Confidentiality
• Operational, administrative and personal identifiers should be
removed from datasets as soon as they are no longer required
to meet the approved purposes of the statistical data
integration project
• When identifiers need to be retained, eg for longitudinal
studies, they should be kept separate from the integrated
dataset
• The number of unit records and data variables to be included
in an integrated dataset should be no more than required to
support the approved purposes
• Eg 5% Statistical Longitudinal Census Dataset, not 100%
Principle 6 cont.
• The type of matching used should be chosen as the minimum
needed to support the approved purposes, and the range of
attributes used to establish a common identity should be the
minimum necessary for the linking operation to succeed
• Eg probabilistic data linking rather than exact data linking
• Access to potentially identifiable data for statistical and
research purposes, outside of secure and trusted institutional
environments, should only occur where : legislation allows; it
is necessary to achieve the approved purposes; and meets
agreements with source data agencies
• Custodians need to advise on legislation
Principle 6 cont.
• Risks of indirect as direct identification should be carefully
managed when data is disseminated outside secure and
trusted environments, particularly of units with unusual
characteristics.
• This management must take account of the potential increase in
identifiability of one set of data when combined with another set.
• It might involve strict data use licensing conditions, reducing
detail, perturbing data, or seeking the consent of the individual or
business involvd to release potentially identifiable data, the last
of these being most likely in the case of business data
Principle 6 cont.
• Once the approved purpose of the project is met, the related
datasets should be destroyed, or if retained, the reasons for
and necessity of retention documented, and a review process
et up. If such a retention was not part of the initial approval
process, re-approval of the decision to retain in required.
• Archiving of statistically integrated datasets should be
restricted to confidentialised datasets.
High Level Principles
www.nss.gov.au
Transparent
Governance
Treat data as a
strategic asset
Public Benefit
Roles
and
Respons
ibilities
Scope
High Level
Principles
Statistical and
Research
Purposes
Custodian’s
retain
accountability
Risks
Privacy and
Confidentiality
Integrating
Authority to
be nominated
Authorisation
Interim
Accreditation
Where is the Journey going
Leadership
Principles
Governance
and
Institutional
Arrangements
?
?
Where is the Journey going
Principle 1 :
Governance
and
Treat data
Leadership
Principles as a
Institutional
strategic assetArrangements
Nationally
Important
Datasets
Where is the Journey going
Principle 1 :
Governance
and
Treat data
Leadership
Principles as a
Institutional
strategic assetArrangements
Nationally
Important
Datasets
Renewed
mandate for
Australia’s
Statistical
System