Contents
1. Introduction
2. Why should NSOs support the research community?
3. Managing access to microdata
4. Supporting legislation and NSO responsibility as
custodian of confidentiality
5. Methods of supporting users
6. Managing tensions between NSOs and researchers
7. Management issues associated with the release of
microdata
8. Microdata: Free or for fee?
9. Copyright issues
1. Introduction
 Growing demand for microdata, resulting
from:
- Increasing data dissemination over the
Internet;
- collaboration among members of the
research community; and
 Access to such data is also of interest to
international agencies for cross-country
comparisons
What is accessibility
Accessibility is the ease with which
statistical data can be obtained by users,
and reflects the availability of information
from the holdings of the agency in
general, and census information in
particular
4
Accessibility is a Census strategic objective
The strategic objectives of a census include:
“implementing policies designed to
safeguard the access of all users to census
results”
PRPHC, Rev 2 - Para 1.16.
5
To whom should microdata be made available?
• Policymakers and researchers employed by line
ministries and planning departments.
• Research and academic institutes involved in social
and economic research.
• Academic staff and students; and
• Other users involved in scientific research, and
• International agencies and other sponsoring agencies.
2. Why should NSOs support the research
community?
 official statistics collected for government and
for the use of the research community.
 research community plays a role in stimulating
policy analysis and debate.
 data are analysed and presented from different
perspectives.
• Leveraging funding for statistics
2. Why should NSOs support the research
community? …. Cont’d
• Enhances the credibility of official statistics
• Improves the reliability and relevance of data
• Reduces duplication in data collection
• Reduces the cost of data dissemination
• Promotes development of new tools for using
data
2.1 The perspective of the NSO regarding
microdata access
• NSOs must maintain the trust of the
respondents , which is fundamental to the
effectiveness of the NSO,
• questions of legal authority, quality and costs
• worry about researchers not being particularly
careful to preserve the validity of their
analysis
• Granting international access sometimes
problem because of legal restrictions.
2.2 The perspective of the research community
(a) access permits policy makers to pose and
analyse complex questions
(b) access to microdata permits analysts to calculate
marginal rather than just average effects.
(c) Access enables replication of important research.
(d) access to microdata, and the resulting feedback,
can facilitate improvements in data quality.
(e) range of outputs increases.
•NSOs can play a very useful role in providing an
accepted and authoritative, as well as high
quality, data source for the research community
thus reducing replication of data collection.
3. Managing access to microdata
“Individual data collected by statistical
agencies for statistical compilation,
whether they refer to natural or legal
persons, are to be strictly confidential and
used exclusively for statistical purposes”
Principle 6 of the Fundamental Principles of Official Statistics
11
3. Managing access - Core principles
Principle 1: It is appropriate for microdata collected for
official statistical purposes to be used for statistical
analysis to support research as long as confidentiality
is protected.
Principle 2: Microdata should only be made available
for statistical purposes.
Principle 3: Provision of microdata should be consistent
with legal and other necessary arrangements that
ensure that confidentiality of the released microdata is
protected.
Principle 4: The procedures for researcher access to
microdata, as well as the uses and users of microdata
should be transparent, and publicly available.
4. Supporting legislation and NSO
responsibility as custodian of confidentiality
At a minimum, release of microdata should be
supported by some form of authority.
- to engender public confidence that there are
legal constraints that determine what can and
cannot be done;
-
to promote mutual understanding between
NSOs and researchers;
-
to provide for greater consistency in the way
research proposals are treated; and
-
to establish a basis for dealing with breaches.
4. Supporting legislation …
• Agreements should be made, not only with
researchers but also with senior officers of
the institutions they represent. Any breaches
should be taken seriously.
5.
Methods of supporting users
 Statistical products for use outside the NSO
•
Statistical Tables and Data Cubes
•
Anonymised Microdata Files (AMFs) - Public Use
Files (PUFs) and licensed files
 A service window through which researchers can
submit data requests - Remote Access Facilities
(RAFs)
 Allowing researchers to work on the premises of
the NSO - Data Laboratories
 Engaging a researcher as a temp NSO staff
6. Managing tensions between NSOs &
researchers
• While researchers often regard some of the
‘controls’ inherent in the microdata access
arrangements as unnecessary bureaucracy, a
breach of confidentiality, have dire
consequences for NSOs.
• If respondents believe or perceive that an NSO
will not protect the confidentiality of their data,
they are less likely to cooperate or provide
accurate data.
6.1 How might the tensions between NSO and
researcher perspectives be resolved?
• rapid expansion of databases means virtually
impossible to completely avoid identification of
persons even though names and addresses are
removed,
• furthermore, technology advances have made data
matching easier
To lessen the tension,
• Transparency is important to avoid accusations of
secrecy;
• Support from champions of use of microdata.
6.2 How do NSOs manage risks of microdata
access?
• Set principles to be followed for access to microdata
• Ensure there is legal and ethical base for protecting
confidentiality.
• Set up an internal committee to debate these matters
and make recommendations to the head of the NSO.
• More access through remote access facilities and data
laboratories if completely unidentifiable microdata for
public release may not be possible
• Explore new technological developments
• Pass some of the onus of responsibility to the research
community.
6.3 How can NSOs pass some of the risk
back to researchers?
• researchers should demonstrate to the NSO the
public benefits of their work and that the
microdata are necessary for this research.
• making them sign a legally binding undertaking
with penalties if they breach confidentiality
provisions.
• ensuring researchers are fully aware of their
obligations through appropriate education.
7. Management issues associated with the
release of microdata
7.1 Managing decision making on confidentiality
It is the chief statistician or his or her delegate who
needs to make the decision on the release of a
microdata file. The chief statistician needs advice
on whether, eg:
(a) the risk of identification is sufficiently small;
(b) the adjustments made to the data items have
not unduly damaged the microdata file; and
(c) variables that have been collapsed are the most
appropriate, taking into account both the needs of
researchers and the identification risk.
7. Management issues …
7.2 Managing metadata
If users are to make effective use of microdata, they
must have access to the appropriate metadata.
This would include:
• a description of the census/survey including any
information on quality;
• a list of the data items and the classifications used
(sometimes referred to as a ‘data dictionary’); and
• definitions of the data items.
7. Management issues …
7.3 Managing breaches by the researcher
• if a legal offence has occurred, legal action should
be considered.
• at a minimum, the researcher should be prevented
from further access to microdata.
• stopping further release to the institution of the
researcher, at least until the institution has taken
appropriate steps in dealing with the offence
committed by the researcher;
• for minor breaches, a warning may be sufficient.
Policies for access to archived data –
Examples
-
South Africa
Ethiopia
Egypt
Botswana
Policies for access to archived data – Example: UK
A guide to putting the principles [on data access and confidentialty] into practice – UK,
Protocol on data access and confidentiality
1. The National Statistician will set standards for protecting confidentiality, including a
guarantee that no statistics will be produced that are likely to identify an individual
unless specifically agreed with them.
2. Respondents will be informed of the main intended uses and access limitations
applying to the information they provide to statistical inquiries.
3. The same confidentiality standards will apply to data derived from administrative
sources as apply to those collected specifically for statistical purposes.
4. Data provided for National Statistics will only be used for statistical purposes.
5. Where information identifying individuals must be given up by law, it will be released
only under the explicit direction and on the personal responsibility of the National
Statistician.
6. Everyone involved in the production of National Statistics will be made aware of their
obligations to protect provider confidentiality and of the legal penalties likely to apply
to wrongful disclosure.
These obligations will continue to apply after completion of service.
7. Data identifying individuals will be kept physically secure.
8. Access (to identifying data) will require authorisation and will only be allowed when
the Head of Profession is satisfied the data will be used exclusively for justifiable
research and that the information is not reasonably obtainable elsewhere.
Policies for access to archived data – Example: IPUMS
1) the NSO retains ownership, including copyright;
2) data are to be used exclusively for statistical purposes associated with teaching, research, and
publishing;
3) use for administrative, commercial or income generating purposes is prohibited;
4) application procedures for obtaining access to microdata are specified in the MOU;
5) confidentiality of the data is protected by means of prohibitions against
a. any attempt to ascertain the identity of individuals, families, households, dwellings or other identities;
b. any allegation that an identification has been made.
In addition there are statements regarding:
6) the necessity of security measures for retaining microdata;
7) publication and citation requirements;
8) procedure for dealing with violations, including sanctions;
9) the sharing of integrated microdata with the National Statistical Offices;
10) recognition of jurisdiction under international law with the ICC International Court of Arbitration for
the settlement of disputes; and
11) establishing the supreme precedence of the MOU over any subsidiary document, contract or other
instrument.
8. Microdata: Free or for a fee?
8.1 Arguments against fee for access to
microdata
• Fees might reduce considerably the number of
potential users and hence the real value of the data.
• In developing countries, it may be an obstacle to
users with the most interest in the data: students,
local research centres, universities, etc.
• Does the fees revert to the NSO or a central
agency?
• Selling imposes an obligation of quality and
service.
• Selling generates little income.
• Most NSO websites are freely available to users
8.2 Arguments supporting fee for access to
microdata
• data producers may consider selling microdata
products as an income-generating activities.
• recover costs incurred in the documentation and
anonymisation process.
• recover other costs such as a review process for
release of files, licensing of data files, operating
a data enclave, supporting users, and
maintaining infrastructure.
• to make beneficiaries of the data service bear
some of these costs.
9. Copyright issues
• microdata copyright interests should be wellestablished by NSO under some kind of legal
instrument consistent with other products
• users should recognize the need for copyright
permission from NSOs.
• exemption to copyright infringement should
be clearly specified , eg if it does not apply to
the government agencies.
• NSOs should clearly inform users what
copyright restrictions, if any, apply to their
microdata and may choose official agreement
that recognizes those rights.
In conclusion,
“A census is not complete until
the information collected is made
available to potential users in a
form suited to their needs” –
including microdata.
(UN Principles and Recommendations for Population and Housing
Censuses, Revision 2, para. 1.206)
References
Principles and Guidelines for Managing Statistical Confidentiality
and Microdata Access; Background document, UN Statistical
Commission, Thirty-eighth session, 27 February - 2 March 2007
IHSN, Dissemination of Microdata Files: Principles, Procedures and
Practices; 2010
Thank you !!