Principles and Practices for a
Federal Statistical Agency
— WHAT AND WHY —
Constance Citro, Director, CNSTAT
COPAFS • June 7, 2013
Washington, DC
“P&P”—A Product of CNSTAT at
the National Academies
1992
2001
2013
2005
2009
2
What Are the National Academies?
National Academy of Sciences—independent,
nonprofit honorific society; 1863
congressional charter to advise government
on science and “art”
National Academy of Engineering—est. 1964
Institute of Medicine – est. 1970
National Research Council – est. 1916 as
operating arm of NAS
About 60 standing units like CNSTAT
3
What Is CNSTAT?
Established in 1972 as a standing unit of the NAS,
recommended by the President’s Commission on
Federal Statistics to provide an independent,
objective resource for evaluation and improvement
of federal statistical methods and operations.
CNSTAT’s mission is to improve the statistical
methods and information on which public policy
decisions are based; it also serves as a coordinating
force in the decentralized U.S. statistical system.
Over its 40-year history, CNSTAT has produced over
240 consensus, interim, and workshop reports.
4
Who Serves on CNSTAT?
LAWRENCE BROWN (chair), statistics
JOHN ABOWD, economics
LISA LYNCH, economics
DAVID CARD, economics
SALLY MORTON, biostatistics
ALICIA CARRIQUIRY, statistics RUTH PETERSON, criminal justice
CONSTANTINE GATSONIS,
TED SHORTLIFFE, bioinformatics
biostatistics
HAL STERN, statistics
JAMES HOUSE, survey research JOHN THOMPSON, survey mgt.
MICHAEL HOUT, sociology
(public/private sectors)*
SALLIE KELLER, statistics
ROGER TOURANGEAU, survey
*on leave of absence
research
Serve pro bono to oversee CNSTAT’s project portfolio
Supported by 14 FTE staff from variety of backgrounds
5
What Does CNSTAT Do and
Who Pays?
Types of work (members of panels et al., are volunteers, reimbursed for travel)
Consensus panels (10–14 members)—review data collection programs, entire
agencies (BJS, BTS, NCES, NCSES, OIS), cross-cutting issues, key measures;
issue reports with findings and recommendations; members are appointed by
NAS president and supported by staff
Stand-alone workshops —convening function, summary report
Other convening activities —planning meetings, seminars, etc.
Principles and Practices —a report authored by CNSTAT as a committee of the
whole
Agency contracts/grants fund individual projects
Over 20 agencies contribute small amounts of core support to let CNSTAT
maintain staff and undertake cross-cutting, “good citizen” activities, such as
P&P: NSF/MMS manages core grant; AHRQ, ASPE, BEA, BJS, BLS, BTS,
Census, CIS/DHS, Defense, EIA, ERS, HUD, NCES, NCHS, NCSES, NEA,
NIA, OIS/DHS, SOI/IRS, SSA
6
Sampling of CNSTAT Studies
7
“P&P”—Five Editions to Date
2001–
44 pp.
1992–
27 pp.
2005—
66 pp.
2013—
166 pp.
2009—
123 pp.
8
What P&P Is and Is Not
It IS a statement of principles that are fundamental and intrinsic to
the concept of a federal statistical agency; AND a statement of
practices that are ways and means of making the basic principles
operational and facilitating an agency’s adherence to them
It IS NOT a guide to best practices for data collection, processing,
estimation, or dissemination; NOR does it provide a set of data
quality standards
It IS a set of guidelines and NOT a set of prescriptions; it has
endeavored to keep up with new challenges and opportunities
It not only applies to federal statistical agencies, but may also be
useful for statistical activities elsewhere, as in other federal
agencies, state/local governments, and other countries
9
Origins – First Edition, 1992
Congressional staff and cabinet departments inquired about what
constitutes an effective statistical agency (per establishing BTS
and contemplating a statistical agency in EPA)
CNSTAT recognized that statistical agencies are sometimes
confronted with inappropriate requests for confidential
information or to put a policy spin on press releases of data
Statistical agencies in our decentralized system must operate under
policies and guidance of many departments of government, and
not all officials are knowledgeable about proper practices for
federal statistics
So, CNSTAT undertook to produce a “white paper,” now known
as “the purple book”
10
Origins, cont’d– 1st Edition, 1992
Some controversy at the time about whether this might better be a
product of OMB
CNSTAT went ahead, involving OMB and statistical agencies in
commenting on a draft
Issued in 1992 on 20th anniversary of CNSTAT’s founding; edited
by Margaret Martin and Miron Straf
Over time, the importance of CNSTAT/NRC/NAS as the
authoring body became clear, given the NAS reputation for
independence and objectivity and that OMB inherently is viewed
as part of an administration
11
Structure – First Edition, 1992
Preface
Acknowledgments
Principles and Practices for a Federal Statistical Agency
[became Part I in subsequent editions; included a definition of a
statistical agency and reasons for establishing same—8 pp. in
1992; 16 pp. in 2013]
Commentary [became Part II in subsequent editions—16 pp. in
1992; 54 pp. in 2013]
References [featured CNSTAT reports, 3 pp. in 1992; 13 pp. in
2013, including FCSM working papers, OMB documents, etc.]
Appendixes [none in 1992; 5 appendixes in 2013, 72 pp.]
12
Definition/Rationale –
First Edition, 1992
Definition of a federal statistical agency:
A federal statistical agency is a unit of the federal government whose
principal function is the compilation and analysis of data and the
dissemination of information for statistical purposes.
Reasons to establish a statistical agency:
Need for information extending beyond narrow scope of individual
operating units
Need to establish independence of major series from policy or
operating control
Need to establish confidentiality protection
Need to consolidate collection, compilation, analysis and dissemination
of statistics in one unit to encourage high-quality performance,
eliminate duplication, and streamline operations
13
Content – First Edition, 1992
Three principles—
Relevance to policy issues
Credibility among data users
Trust among data providers and data subjects
Eleven practices—
A clearly defined and well-accepted mission
A strong measure of independence
Fair treatment of data providers
Cooperation with data users
Openness about the data provided
Commitment to quality and professional standards
Wide dissemination of data
An active research program
Professional advancement of staff
Caution in conducting nonstatistical activities
Coordination with other statistical agencies
14
“Strong Measure of Independence”
Arguably, most important but also trickiest concept in P&P
Tension: Agencies must operate within framework of
congressional, OMB, and departmental oversight, but also
maintain credibility as impartial purveyors of information
Aspects of independence cited in 1st edition:
• Organizational separation from department program activities
• Professionally qualified head; appointment for a fixed term;
direct access to the secretary
• Broad authority over scope, content, and frequency of data
• Primary authority for selection and promotion of prof. staff
• Authority to release statistical info without prior clearance
• Authority to speak on agency’s program before Congress, with
congressional staff and before public bodies
• Adherence to predetermined schedules for release of key stats
• Maintenance of clear distinction between statistical releases and
policy interpretations put out by others
15
Origins – Second Edition, 2001
First edition used by federal statistical agencies to inform
department officials, advisory committees, and others
Used in a U.S. GAO (1995) study of the performance of the major
statistical agencies
Used by Janet Norwood in her 1995 book, Organizing to Count
Informed establishment and later assessment of BTS (NRC, 1997)
Cited in confirmation hearings (e.g., NCES commissioner, 1999)
CNSTAT decided that it would be useful to release a revised and
updated version and, moreover, to do so every 4 years at the
beginning of a new administration or second term
Second edition editors were Margaret Martin, Miron Straf, and
Connie Citro
16
Changes – Second Edition, 2001
Appendixes reproduced Fundamental Principles of Official Statistics of
the Statistical Commission of the UN (adopted 1994) and provided
list of selected federal statistical web sites
Added explicit discussion of the need for federal statistics:
Statistics that are publicly available from government agencies are essential
for a nation to advance the economic well-being and quality of life of its
people . . . . Federal statistical agencies are established to be a credible source
of useful, accurate statistics in one or more subject areas that are available to
the public and policy makers on a timely basis . . . . Private-sector
organizations also provide useful statistical information, . . . but private
markets are not likely to provide all of the data that are needed . . . or to
make data as widely available as needed . . . .
Same 3 principles; practices rearranged somewhat
17
Content – 2nd Edition, 2001
Three principles—
Relevance to policy issues
Credibility among data users
Trust among data providers and data subjects
Eleven practices—
A clearly defined and well-accepted mission
A strong position measure of independence
Continual development of more useful data [recognizing Internet era]
Openness about the data provided
Wide dissemination of data
Cooperation with data users
Fair treatment of data providers
Commitment to quality and professional standards of practice
An active research program
Professional advancement of staff
Caution in conducting nonstatistical activities
Coordination and cooperation with other statistical agencies
18
Changes – Third Edition, 2005
Added appendix on Relevant Legislation and Guidelines [for federal
statistical agencies] Adopted Since 2001:
• CIPSEA (Confidential Information Protection and Statistical
Efficiency Act, aka Title V of E-Government Act of 2002)
• Privacy impact assessments (PIAs) (Section 208 of E-Govt. Act)
• Information Quality Act of 2000
• OMB guidance on peer review
• Performance assessment rating tool (PART)
Added as a characteristic of statistical agencies that strengthens
independence: authority [over] . . . information technology systems [to]
maintain the integrity and confidentiality of data and reliably support timely
and accurate production of key statistics
[sent letter to OMB on this topic in 2003]
19
Content – Third Edition, 2005
Three principles—
Relevance to policy issues
Credibility among data users
Trust among data providers
Eleven practices—
A clearly defined and well-accepted mission
A strong position of independence
Continual development of more useful data
Openness about sources and limitations of the data provided
Wide dissemination of data
Cooperation with data users
Fair treatment of data providers
Commitment to quality and professional standards of practice
An active research program
Professional advancement of staff
Coordination and cooperation with other statistical agencies
20
Changes – Fourth Edition, 2009
Appendixes added on Organization of the Federal Statistical System
[provides brief histories of major statistical agencies plus
examples of other statistical programs] and Legislation and
Regulations That Govern Federal Statistics:
• Paperwork Reduction Act (PRA)
• 2000 Information Quality Act and guidelines
• 1997 OMB order providing for confidentiality of statistical info
• 2002 CIPSEA and 2007 guidance
• 2002 E-Government Act, Section 208, PIAs
• 2004 OMB peer review guidance
• 2002–2008 OMB PART
• OMB statistical policy directives (metro areas, race/ethnicity,
industry/occupation codes, survey standards, etc.)
21
Cabinet chart
22
Related
Agencies
Layering chart
Congressional
Appropriations
Subcommittee
Agriculture,
Rural
Development,
Food and Drug
Administration,
and Related
Agencies
Commerce,
Justice,
Science, and
Related
Agencies
Energy and
Water
Development
Financial
Services and
General
Government
Interior,
Environment,
and Related
Agencies
Labor, Health
Financial
Services and
General
Government
Reporting Authority, Parent Department/Agency to
Principal Statistical Agency
Secretary of
Agriculture /
Deputy
Under
Secretary for
Research,
Education,
and
Economics
Economic Research
Service
Secretary of
Commerce /
Deputy
Under
Secretary for
Economic
Affairs,
Economics
and Statistics
Administration
Bureau of
Economic Analysis
Attorney
General /
Deputy
Associate
Attorney
General
National Science
Foundation
Director /
Deputy
Social,
Behavioral,
and Economic
Sciences
Directorate
Secretary of
Energy / Deputy
Energy
Information
Administration
Energy and
Water
Development
Interior,
Environment,
and Related
Agencies
National Agricultural
Statistics Service
Attorney
General /
Deputy
Associate
Attorney
General
National Science
Foundation
Director /
Deputy
Social,
Behavioral,
and Economic
Sciences
Directorate
Secretary of
Energy / Deputy
Energy
Information
Administration
Office of
Management
and Budget
Director
Deputy
Director for
Management
Office of
Information
and
Regulatory
Affairs
Secretary of the
Treasury /
Deputy
Commissioner,
Internal
Revenue
Service
Office of
Research,
Analysis, and
Statistics
Environmental
Protection
Agency
Administrator /
Deputy
Assistant
Administrator
and Chief
Information
Officer
Office of
Environmental
Information,
Environmental
Protection
Agency
Secretary of
Labor / Deputy
Bureau of
Labor Statistics
Secretary of
Education /
Under Secretary
Director,
Institute of
Education
Sciences
Secretary of
Health and
Human
Services /
Deputy
Director,
Centers for
Disease
Control and
Prevention
Commissioner of
Social Security /
Deputy
Deputy
Commissioner
for Retirement
and Disability
Policy
Office of Research,
Evaluation, and
Statistics,
Social Security
Administration
Secretary of
Transportation /
Deputy
Director,
Research and
Innovative
Technology
Administration
(RITA)
Bureau of
Transportation
Statistics
General,
Office of
Justice
Programs
Bureau of
Justice Statistics
National Center for
Science and
Engineering
Statistics, National
Science Foundation
Statistical and
Science Policy
Office, Office of
Management and
Budget
Statistics of Income
Division, Internal
Revenue Service
Census Bureau
Assistant
Attorney
General,
Office of
Justice
Programs
Bureau of
Justice Statistics
National Center for
Science and
Engineering
Statistics, National
Science Foundation
Office of
Management
and Budget
Director
Deputy
Director for
Management
Office of
Information
and
Regulatory
Affairs
Secretary of the
Treasury /
Deputy
Commissioner,
Internal
Revenue
Service
Office of
Research,
Analysis, and
Statistics
Environmental
Protection
Agency
Administrator /
Deputy
Assistant
Administrator
and Chief
Information
Officer
Office of
Environmental
Information,
Environmental
Protection
Agency
Secretary of
Labor / Deputy
Bureau of
Labor Statistics
Secretary of
Education /
Under Secretary
Director,
Institute of
Education
Sciences
National Center for
Education Statistics
Statistical and
Science Policy
Office, Office of
Management and
Budget
Labor, Health
and Human
Services,
Education, and
Related
Agencies
Transportation,
Housing and
Urban
Development,
and Related
Agencies
National Center for
Education Statistics
Coordinating
Center for
Health
Information
and Service
National Center for
Health Statistics
Statistics of Income
Division, Internal
Revenue Service
23
Content – Fourth Edition, 2009
Four principles—
Relevance to policy issues
Credibility among data users
Trust among data providers
A strong position of independence [moved from practice per agencies]
Eleven practices—
A clearly defined and well-accepted mission
Continual development of more useful data
Openness about sources and limitations of the data provided
Wide dissemination of data
Cooperation with data users
Fair treatment of data providers
Commitment to quality and professional standards of practice
An active research program
Professional advancement of staff
A strong internal and external evaluation program
Coordination and cooperation with other statistical agencies
24
Uses of Second–Fourth Editions
Third Edition cited by GAO in Bureau of Justice Statistics: Quality
Guidelines Generally Followed for Police-Public Contact Surveys, but
Opportunities Exist to Help Assure Agency Independence, March 30,
2007 [question concerned press releases that accompany data
releases]
Third Edition cited in OMB Statistical Policy Directive No. 4—Release
and Dissemination of Statistical Products Produced by Federal Statistical
Agencies, March 7, 2008
Fourth Edition endorsed by ASA Board, April 30, 2009
Fourth Edition cited as key document in Statement of Commitment to
Scientific Integrity by Principal Statistical Agencies
Series cited by GAO in Agencies Can Make Greater Use of Existing
Data, but Continued Progress Is Needed on Access and Quality Issues,
February 2012
25
Changes – Fifth Edition, 2013
Included Dedication to Margaret Martin, life-time COPAFS
board member, who died at age 100 in May 2013
Added Executive Summary (available separately as Highlights)
Added European Statistics Code of Practice for the National and
Community Statistical Authorities (adopted 2011)
To appendix on legislation/regulations, added:
• 2010 Government Performance and Results Modernization Act
• 2009–2010 Guidance on Scientific Integrity
• 2002 Federal Information Security Management Act (FISMA)
Beefed up Part II Commentary on Continual Development of More
Useful Data to discuss surveys, admin records, and other sources
Beefed up discussion of Independence and nonstatistical activities
Separated Privacy from Confidentiality Protection
Stressed need for active Collaboration among agencies
26
Content – Fifth Edition, 2013
Four principles—
Relevance to policy issues
Credibility among data users
Trust among data providers
Independence from political and other undue external influence*
A strong position of independence
*Footnote (p. 14)
A statistical agency actively works to obtain a broad range of
external input to develop its programs: “undue external influences”
are those that seek to undermine an agency’s impartiality and
professional judgment.
27
Content – Fifth Edition, 2013
Thirteen practices—
A clearly defined and well-accepted mission
Necessary authority to protect independence
Continual development of more useful data
Openness about sources and limitations of the data provided
Wide dissemination of data
Cooperation with data users
Respect for the privacy and autonomy of data providers
Protection of the confidentiality of data providers’ information
Fair treatment of data providers
Commitment to quality and professional standards of practice
An active research program
Professional advancement of staff
A strong internal and external evaluation program
Coordination and collaboration cooperation with other statistical agencies
28
QUESTIONS? COMMENTS?
(I am readying a File for the
6th Edition of P&P)
Constance Citro
Director, Committee on National Statistics
ccitro@nas.edu • (202) 334-3009
29