Statistics New Zealand's Microdata Access Protocols Introduction As part of its mission to “provide New Zealanders with a national statistical service of integrity that is valued and trusted for publishing useful information” Statistics New Zealand wants the best possible use made of the data it collects. Microdata access is an increasingly important input to social science and economic research. Statistics New Zealand provides researchers with access to its statistical microdata, provided they meet specific obligations on confidentiality that come from the Statistics Act 1975. All access to microdata is provided under clear and strict conditions that are closely monitored to ensure that there is no breach of confidentiality. Microdata is defined as unit record or low level data that has not been sufficiently aggregated to be made publicly available. Synthetic or quasi unit record datasets, which require Statistics New Zealand to have regard to similar issues of confidentiality, will also be considered within the context of these protocols. Statistics New Zealand has established these protocols to facilitate the use of unit record data for research while ensuring obligations to protect the confidentiality of information supplied by respondents are met. They describe the basis on which proposals for access to microdata are assessed and outline the conditions which may be imposed as part of any access agreement. 1. Statistics Act provisions It is important for all researchers to be familiar with the confidentiality provisions of the Statistics Act 1975, as well as other relevant legislation such as the Privacy Act. Relevant extracts from the Statistics Act are included as Appendix A. There are four sections of the Statistics Act which make it possible to provide access to microdata researchers. Government departments Under Section 37C of the Act, individual schedules (i.e. microdata) can be disclosed, in certain circumstances, to other Government Departments. This disclosure is at the discretion of the Government Statistician and is "solely for bona fide research or statistical purposes pursuant to the functions and duties of that Government Department". Employees and researchers who are contracted to a government department who require access to microdata must sign a declaration of secrecy. They are then considered to be employees of Statistics New Zealand for the purposes of implementing the security and confidentiality provisions of the Statistics Act. Researchers contracted to Statistics New Zealand Under Section 21(3B), an independent contractor under contract to Statistics New Zealand, and any employee of that contractor, can be required by the Government Statistician to make a statutory declaration of secrecy. They are then considered to be employees of Statistics New Zealand for the purposes of the security and confidentiality provisions of the Statistics Act. This provision enables Statistics New Zealand to provide researchers with access to microdata and requires them to protect the information in the same way as employees of Statistics New Zealand are obliged to do. Such access is granted to researchers undertaking work considered by Statistics New Zealand to fit with the purpose of Statistics New Zealand and which is of significant public interest. Joint surveys Under Section 9, information that is collected jointly by Statistics New Zealand and another Government Department, local authority or statutory body can be exchanged between those organisations. Employees from the other organisations involved in collecting or processing the data are required to sign a declaration of secrecy. Obligations under the Act will extend to any agency involved in a joint collection. Permission from supplier Under Section 37 information may be disclosed if consent is obtained in writing from the person supplying it. These protocols mainly consider the first two types of access – by government departments and by researchers contracted to Statistics New Zealand. Proposals to access microdata under other provisions will be assessed using the principles outlined here. In some cases, Statistics New Zealand may hold datasets which are not collected under the Statistics Act, however relevant provisions of the Act, as well as appropriate policies, will be considered to apply to them. Penalties for contravening the Statistics Act are outlined in the Act. 2. Methods of access Two forms of microdata access are available, either: the researcher analyses datasets while in a protected environment on Statistics New Zealand premises (on-site access), or the researcher has custody of a dataset for analysis (off-site access). The guiding principle here is one of "safe custody". Statistics New Zealand must be assured that the data is going into safe custody before access is granted. This is done by either controlling the environment within which the data is accessed or by allowing it to go into the custody of an agency or an individual who will maintain the protection of confidentiality that Statistics New Zealand pledges to its respondents. Clearly the latter involves greater risk to Statistics New Zealand’s relationship with its respondents. Off-site access will only be permitted in special cases, where: the researcher is an employee of a government department (or a researcher contracted to the department) which has no conflicting role relating to the use of the data, and the agency is abiding by the Protocols for Official Statistics; or explicit consent has been granted by each respondent for the release of their data; or there are special circumstances which the Government Statistician will consider on a caseby-case basis. Population census data and integrated datasets will not be approved for off-site use under any circumstances. On-site access is provided through the Data Laboratory in a designated space on Statistics New Zealand premises. The conditions of use of the Data Laboratory are attached as Appendix B. The general principles that Statistics New Zealand expects an agency to follow to implement a secure environment for off-site use of unit record statistical datasets supplied by Statistics New Zealand are attached as Appendix C. 3. Proposed research Microdata access is provided by Statistics New Zealand to allow authoritative statistical research of benefit to the public of New Zealand. It is provided at the discretion of the Government Statistician. To enable the Government Statistician to make a decision a detailed proposal is required. A research proposal must provide details on the aims and outcomes of the proposed research. Details must be provided on proposed analytical tools; this includes software to be used, as well as the methodology. The aim is to ensure that output is accessible and easily understood by the Statistics New Zealand staff assigned to check that outputs do not disclose individual information. A research proposal must demonstrate why there are no satisfactory alternatives to using microdata. Consistent with the Statistics Act, the research must be based on unidentified records and outputs should not disclose any identifiable information. A copy of all output made publicly available should be provided to Statistics New Zealand. a. Government departments A research proposal to obtain access to a dataset must demonstrate that it is consistent with the Statistics Act requirement that the use is “solely for bona fide research or statistical purposes pursuant to the functions and duties of that department”. Government departments must seek approval for a dataset in their custody to be used for a purpose that differs from that approved. In most cases it is expected that departments will make output from the research publicly available. Where this is not intended, departments need to be specific as to the proposed use. All outputs produced by government departments will be subject to Crown Copyright administered by that department, including the right to authorise the reproduction and dissemination of outputs. b. Researchers contracted to Statistics New Zealand Access to data that is approved for one purpose cannot be used for another purpose without the prior agreement of Statistics New Zealand. Any research undertaken by a researcher, contracted to Statistics New Zealand for the purpose of that research, must be made publicly available. Researchers are expected to provide specific information about their plans to publish output. Statistics New Zealand may require it to be subject to independent peer review. Statistical data summaries created by researchers contracted to Statistics New Zealand belong to Statistics New Zealand. Statistics New Zealand will not provide access to microdata collected under the Statistics Act for commercial gain. 4. Researchers All persons who are provided with access to microdata are required to sign the statutory declaration of secrecy as specified in the Statistics Act. a. Government departments An agreement will be negotiated between Statistics New Zealand and the department which sets out the conditions of access. Government departments who have been granted approval for access to microdata must ensure that employees and contracted researchers, who require access to the microdata, are aware of their obligations under the Statistics Act and the Protocols for Official Statistics. Statistics New Zealand may require documentation on the credentials of researchers contracted to a department for the purposes of undertaking research involving access to microdata. A signed statutory declaration of secrecy must be forwarded to Statistics New Zealand prior to the provision of access to an employee or a contracted researcher. Where a researcher is working for an agency which maintains a person, dwelling or business register or administration records, there is, at least in perception, a potential for the microdata to be linked with such data sources and hence a greater disclosure risk. Assurances will need to be given by the chief executive of such an agency that any such statistical research is undertaken separately from any regulatory function and no linking of records will be undertaken. b. Researchers contracted to Statistics New Zealand Access to microdata will be provided to reputable researchers who have a proven track record of producing authoritative research in the public domain, or to researchers who are under the supervision of such persons. Statistics New Zealand will require documentation on the credentials of researchers applying to have access to microdata. Researchers, and where possible their supporting institution, e.g. university faculty or department, will be expected to sign an agreement embodying the conditions of access. Any person who requires access to microdata must be included in an agreement. A researcher cannot transfer the rights of access to anyone not listed in the agreement. 5. The data to be accessed All microdata that Statistics New Zealand holds is considered to be eligible for access under this protocol. However, other policies and protocols may need to be consulted. Statements made at the time the data was collected will be taken into account in the assessment of any proposal. Such statements may go beyond what is stated in the Statistics Act and in some cases may restrict access. Generally: The purpose for which the data is collected and any research proposal need to be consistent. Statistics New Zealand will provide access to the subset of the data necessary to undertake the research. Statistics New Zealand will determine the appropriate level of protection required by the data. Statistics New Zealand will provide researchers with access to available meta-data relating to the requested dataset to enable effective use of the data. This will include variable code-lists, the coverage of the survey, and information about the quality of the data (e.g. non-response, sampling and non-sampling error studies, etc). Statistics New Zealand will supply any caveats that relate to the dataset and researchers may be required to include these with any output. Some datasets held by Statistics New Zealand have been collected under an Act other than the Statistics Act. Consent to access to such microdata must be obtained from the Chief Executive of the steward agency. Access must take into account provisions in the Privacy Act which govern the use of information beyond the purpose for which it was originally collected. Examples of such data sources are vitals, imports and exports, building consents data and justice statistics. The Privacy Commissioner will be consulted in cases where an express legal provision exists that parallels or constrains the Statistics Act. Under section 37 of the Statistics Act, respondents may have been explicitly asked whether they would allow the release of their data to other agencies or researchers. However all the issues outlined in these protocols must still be assessed to ensure that relationships with the suppliers of data are protected. Where a researcher proposes linking two or more datasets the proposal will be assessed under the Data Integration protocols (doclink to come). If the proposed linking includes use of the population census, the population census record-matching policy (doclink) will also be consulted. Requests for access to microdata from integrated datasets will be referred to steward agencies of the source datasets for approval. 6. Methods of minimising disclosure risk Statistics New Zealand will take appropriate steps to minimise the potential disclosure risks associated with access to microdata to ensure that it meets its obligations under the Statistics Act to protect respondent’s data. Research proposals must specify the variables that are required and access will usually be restricted to those variables. For some variables, that are more likely to lead to disclosure, the department has established a maximum level of detail that will be provided in a dataset. Requests to access data in more detail will be considered on a case by case basis. It may be necessary to make compensating adjustments to other variables to avoid an increase to disclosure risk overall. Researchers may be limited to a sample of records from any full census dataset such as the population census. Direct identifiers such as names and addresses, where recorded, will be removed from the dataset. Statistics New Zealand will check and, where necessary, modify datasets before they are provided to researchers to minimise the risk that researchers can identify individual respondents. Researchers will be advised of any steps that have been taken. Statistics New Zealand is obligated under the Statistics Act to protect all data no matter how old it is, however it is recognised that the risks of disclosure may diminish with age and this will be taken into account when deciding on access. All output will be reviewed to ensure there are no confidentiality breaches. The published results of any research must not divulge any more information than the Government Statistician would publish and must comply with any policies and rules that Statistics New Zealand has developed to give effect to the confidentiality provisions of sections 37 and 37C of the Statistics Act. Statistics New Zealand reserves the right to issue any other rule to protect confidentiality. Specific confidentiality rules may need to be implemented for more complex datasets, e.g. integrated datasets, on a case by case basis. Output produced using microdata must be scrutinised to ensure it meets the relevant confidentiality requirements before it is released to any person who has not signed the statutory declaration of secrecy specified in the Statistics Act. In cases where the analysis is to occur in the Data Laboratory output will be examined prior to removal from Statistics New Zealand to ensure it meets confidentiality requirements. Where approval has been granted for off-site access, employees, who are authorised to access the data, must ensure output meets confidentiality requirements before it is provided to anyone else. A copy of all output made publicly available should be forwarded to Statistics New Zealand. Appendix A – Relevant Statistics Act extracts These extracts are provided as a guide only. Statistics Act in full Researchers are expected to consult the Definition from s2 Interpretation - "Schedule" means any book, document, form, card, tape, disc, or storage media on which the information required is entered or recorded or is required to be entered or recorded for statistical purposes under this Act: 21. Declaration of secrecy(3B) Every independent contractor engaged by the Department under any contract (other than a contract of employment), and every employee of any such independent contractor, who is likely, in the course of carrying out the terms of that contract, to have access to information collected under the authority of this Act shall, if required to do so by the Statistician, make a statutory declaration of secrecy similar to the declaration of secrecy prescribed by subsection (1) of this section, regardless of whether that independent contractor or employee has made a declaration of secrecy under any other Act. (3C) A person who has made a statutory declaration of secrecy under subsection (3B) of this section shall be deemed to be an employee of the Department for the purposes of(a) Subsection (1) of this section; and (b) Subsections (2) and (3) of section 37 of this Act. 37C. Disclosure of individual schedules to other Government Departments for bona fide research or statistical purposes(1) Notwithstanding section 37 of this Act, the Statistician may disclose individual schedules to any officer of another Government Department solely for bona fide research or statistical purposes pursuant to the functions and duties of that Government Department. (2) No individual schedule shall be disclosed pursuant to this section unless(a) The name and address of the person or undertaking by whom the schedule was supplied is deleted; and (b) Every person involved in the research or statistical project makes a statutory declaration similar to the declaration of secrecy prescribed by section 21 of this Act; and (c) The Statistician is satisfied that the security of the schedules and any information contained in them will not be impaired. (3) The published results of any such research or statistical project shall not divulge any more information than the Statistician could publish under this Part of this Act. (4) Every officer of a Government Department to whom any individual schedule is disclosed pursuant to this section, and that Government Department, shall comply with any directions given by the Statistician relating to the schedule and the information contained in it. Appendix B – Conditions of use of Statistics New Zealand’s Data Laboratory Statistics New Zealand is legally required to protect confidential individual and corporate information. In granting access to microdata for bona fide research or statistical purposes under the Statistics Act 1975, the following conditions cover the use of the Data Laboratory. Access: A designated workstation will be provided. No electronic equipment, e.g. laptop, other than that designated is to be used in the Data Laboratory. Access to a telephone will be provided. There must be no attempt to access unauthorised parts of Statistics New Zealand’s computing environment, or unauthorised areas of Statistics New Zealand’s office space. The Data Laboratory is available during standard office hours (8.30am - 4.30pm) unless otherwise agreed. Bookings must be made 24 hours in advance. Bookings must be for half or full day sessions (morning: 8.30am-12 noon, afternoon:1.00pm - 4.30pm). If an authorised Data Laboratory supervisor is unavailable, access will be refused. Where there is high demand the Data Laboratory manager reserves the right to allocate sessions to achieve balanced access. Access will not be provided to the Data Laboratory to anyone not specified in an agreement and who has not signed the Declaration of Secrecy. Any material (information, equipment, etc.) that is brought in or removed from the Data Laboratory must be declared to the Data Laboratory supervisor. Computing facilities (subject to change): Pentium PC 200MHz or higher Standalone or access to network - If standalone, only the approved dataset, software and results may be stored on the PC - If network access is approved, a personal userid will be created and access restricted to specific directories Use of a nearby laser printer External disk drives locked off and diskettes must be virus checked before each use Software supplied by Statistics New Zealand (subject to change): Standard Microsoft Windows 95 Microsoft Office 97 (Word, Excel, Powerpoint) Optional Microsoft Access SAS Supercross Software provided by the user: Licensing of any software provided by the user is the responsibility of the user. Externally provided software must be loaded by Statistics New Zealand staff. Software must be removed by Statistics New Zealand staff. Output: All output, whether in hard copy or electronic format, must be checked and cleared by an authorised staff member before it can be removed from the Data Laboratory. If the staff member authorised to check output is unavailable, all output must be left behind for checking and forwarding on. Statistics New Zealand undertakes to clear this output within 24 hours, unless otherwise agreed. Other output may be stored in the Data Laboratory until no longer required and then disposed of in the allocated facility. Statistics New Zealand reserves the right to amend these conditions. Appendix C – Secure Environment Guidelines These guidelines set out the principles that Statistics New Zealand expects an agency to follow to implement a secure environment for unit record statistical datasets supplied by Statistics New Zealand for use on an agency’s own site. Use of unit record datasets is governed by the Statistics Act 1975, as well as by other conditions determined by the Government Statistician. Unit record datasets may be accessed only by authorised staff. Authorised staff are those: whose names are advised to Statistics New Zealand as requiring access for the purposes described in the Memorandum, who understand their obligations in respect of confidentiality as set out in the Statistics Act 1975, and who have signed the statutory declaration of secrecy required under the Statistics Act 1975. Agencies must take all reasonable steps to provide a safe environment for unit record datasets for the period of the Memorandum. A safe environment means one in which: a deliberate attempt by an unauthorised person (whether external or internal) is extremely unlikely to succeed, unintentional access by an unauthorised person is unlikely, unnecessary access by systems staff, who may have the ability to access a dataset but not the need, is controlled The agency must appoint an employee to be responsible for developing, implementing and maintaining security measures to give effect to the requirements of the Statistics Act 1975 and the Government Statistician. Examples of appropriate security measures are: password protection with passwords changing regularly encrypted storage secure PCs, standalone networks code of practice for operations staff internal authorisation process for access internal confidentiality checking process An annual update must be provided identifying any changes in the environment.