UNSD/ NSCB Regional workshop on data dissemination & communication
Manila, Philippines, 20-22 June 2012
Promoting (survey) microdata
dissemination policies:
the experience of the IHSN / ADP
François Fonteneau, OECD/PARIS21
International Household Survey Network (IHSN)
Accelerated Data Program (ADP)
francois.fonteneau@oecd.org
IHSN / ADP: background
• Marrakech Action Plan for Statistics (MAPS): creation of IHSN and ADP
IHSN: to improve the coordination of
intl. organisations’ household survey
activities.
ADP: to help data producers in
developing countries increase
the use and value of their survey data.
• Program funded by World Bank, implemented by a small core team
(P21/WB + others); open partnership (ADB, UNSIAP, SPC, DFID , FAO, others)
Survey microdata: growing supply,
unsatisfied demands
By producers:
• “we don’t know how to deal with microdata”
– how to document them?
– how to improve transparency on sources?
– can we disseminate them in our country?
• If YES, how to disseminate them?
• To whom?
• What costs?
• “what do others do?”
– identification of good practice
Survey microdata: growing supply,
unsatisfied demands
By users:
•
•
•
•
•
•
•
“we didn’t know they had data”
Visibility
“they don’t want to share them”
Accessibility
“these data are messy”
Reliability
“this is not really what I need”
Relevance
“how can I assess progress?”
Comparability
“where is the documentation?”
Usability
“there is no transparency” Quality, credibility
Different needs, different challenges
Reports, tables (PDF)
Any web development
tool
Indicators
CensusInfo/DevInfo, etc.
On-line interactive
tabulation, analysis and
mapping tools
REDATAM/IMIS, SuperStar,
Nesstar, etc.
Microdata (n% sample) and
detailed metadata
IHSN MMT + NADA
What outputs?
What tools?
No issue of
confidentiality
Disclosure risk
managed by the
application
Requires data
anonymization
Promoting microdata dissemination
policies: a variety of challenges
•
•
•
•
•
Legal
Strategic
Organizational
Financial
Technical
apply the law…or change it
set your priorities (NSDS)
break silos
allocate minimal budget
steal good practice
• … and cultural (fear of change)
=> Advocacy on benefits / risks
The importance of documentation
For discovering data
– You and your users will know about data availability
by providing searchable data catalogs
For making data usable
– Users need to fully understand the data: why, by
whom, when, and how they were collected,
processed and disseminated
For assessing data quality
=> International metadata standards for microdata:
DDI
DDI : before and after
(...or worse !)
GSBPM
GSBPM, DDI, SDMX
IHSN toolkit for documentation
IHSN Microdata Management Toolkit (DDI metadata editor)
NADA: on line searchable survey cataloguing system
Confidentiality of microdata
• Anonymized datasets for dissemination
– Anonymization techniques/tools:
• Risk measurement / risk reduction / information loss
• Methods: recoding, rank swapping, micro-aggregation,
sampling, etc.
• Available tool (R package): sdcMicro {guidelines in
preparation}
– Different levels of anonymization / different users
controls / different procedures : PUF; SUF
• Full dataset in enclave  direct control of NSO
Confidentiality of microdata
• Only when no confidentiality
concerns
• 3 modes for file distribution:
– PUF - Direct
– SUF - Licensed
– Enclave
• Screen users and usage
• Progressive approach
Principles for microdata dissemination
From UNECE Task Force
1. It is appropriate for microdata collected for official statistical purposes
to be used for statistical analysis to support research as long as
confidentiality is protected (i.e. as long as it is not possible to identify
data referring to an individual).
2. Microdata should only be made available for statistical purposes.
3. Provision of microdata should be consistent with legal and other
necessary arrangements that ensure that confidentiality of the
released microdata is protected.
4. The procedures for researcher access to microdata, as well as the
uses and users of microdata should be transparent, and publicly
available.
UPDATES IN NATIONAL STAT LEGISLATION?
Principles, guidelines and practices
Formulating an access policy and procedures
http://www.ihsn.org/home/index.php?q=focus/disseminationmicrodata-files-principles-procedures-and-practices
PHILIPPINES
Good practice !
REQUEST ACCESS TO
DATA !
INDIA
COLOMBIA
VANUATU
More?
• IHSN: www.ihsn.org
• ADP: www.ihsn.org/adp
• francois.fonteneau@oecd.org
• odupriez@worldbank.org
THANK YOU !