Revised, October 9, 2014
Institutional Processes for Maintaining User-Producer Partnerships in the United States
By Robert P. Parker, Consultant on Federal Statistics
5500 Friendship Blvd
Chevy Chase, MD 20815 USA
Telephone 301-467-7677; fax 301-263-0645
E-mail Parkerrobertp@aol.com
Contributed Paper Session “User-Producer Partnerships,” October 8, 2014
Abstract
This paper describes how the United States statistical system builds partnerships with
major users of their programs. Outreach efforts of individual statistical agencies
combined with the activities of professional organizations and associations representing
the private and research/academic communities, and interagency activities that make
the system more efficient, as well as laws that support these actions, all contribute to a
statistical system that works to serves its users. However, as in most countries, elected
officials determine the scope of statistical programs. The major law is the Paperwork
Reduction Act, which specifies what agencies must do to collect data and requires public
input. Professional organizations and associations and their consortiums meet regularly
with statistical agencies to provide input on ongoing and proposed programs. Agencies
solicit input from formal advisory committees, public requests, and from contracts with
the Committee on National Statistics. Within the statistical system, the Executive Office
of the President is responsible for implementing rules for obtaining public input, for
coordinating the activities of statistical agencies, and for developing government-wide
standards such as the classification of industries. In addition, the paper provides
examples on how statistical agencies obtained public input for several major programs.
Keywords
Principles for Statistical Agencies, Outreach, President, Congress, Office of
Management and Budget, Paperwork Reduction Act, Advisory Committees, Committee
on National Statistics, Professional Organizations and Associations, American
Community Survey, Compass Points Handbooks, Consumer Expenditure Survey,
Gemini Project.
1
1. Introduction
This paper discusses partnership building by United States (US) statistical agencies with
major users of their programs. It traces the origins of the legislation that requires
agencies to obtain input and describes the current requirements. It also covers user
outreach efforts of individual statistical agencies, the activities of professional
organizations and associations that represent private and research/academic
communities, and interagency activities. The paper recognizes that the scope and nature
of statistical programs are largely determined by actions of elected officials – the
President, who proposes the programs, and the legislature (Congress), that approves
and funds them. Recent actions by the Congress have emphasized its role as legislative
actions have tried to eliminate programs, redesign survey forms, or change program
methodology usually without regard to the wishes of users.
The paper includes two examples of the extent of outreach efforts for two major
statistical programs – the American Community Survey (ACS) and the Consumer
Expenditure Survey (CE). It concludes with some observations on the effectiveness of
users’ input on the US federal statistical system.
2. Background on User Input
The importance of user input into the scope and design of statistical programs has long
been stressed as a key practice for statistical agencies by both international and national
organizations.
For the US, in May 2014, the President’s Office of Management and Budget (OMB), the
office in the executive branch responsible for overseeing the federal statistical system,
proposed a new statistical policy directive, “Fundamental Responsibilities of Federal
Statistical Agencies and Recognized Statistical Units.” This directive was based largely
on Principles and Practices for a Federal Statistical Agency, Fifth Edition, published in
2013 by the Committee on National Statistics (CNSTAT) of the private National
Academies of Science (NAS). NAS is a private nonprofit organization of distinguished
scholars engaged in scientific research and was chartered by Congress in 1863 to
advise the federal government on scientific and technical matters. The proposed OMB
directive, by providing key responsibilities for a statistical agency, supports agency
decision-making and enhances data quality. Its framework requires statistical agencies
to adopt policies, best practices, and procedures to implement the responsibilities.
The CNSTAT report used as the basis for the OMB proposal recommended four
“principles” for federal statistical agencies: Relevance to Policy Issues, Credibility Among
Data Users, Trust Among Data Providers, and Independence from Political and Other
Undue External Influence. The first principle is related to user input and states in part:
To establish priorities for its programs, a statistical agency must not only work
closely with the executive branch, Congress, and interested non-governmental
groups, but also engage a broad spectrum of users in the business sector,
academia, state and local governments, and elsewhere. Interaction with
2
stakeholders is essential to enable a statistical agency to continually reassess
the needs of its users for information.
It should be noted that these CNSTAT reports, the first was issued in 1992, have no
official status. Nevertheless, most official statistical agencies operated under its
recommendations and used them to explain to elected officials their need for
independence.
The OMB proposal slightly modified the CNSTAT report. In place of the four “principles,”
the new directive spelled out four “responsibilities:” Produce and Disseminate Relevant
and Timely Information, Conduct Credible and Accurate Statistical Activities, Conduct
Objective Statistical Activities, and Protect the Trust of Information Providers by
Ensuring the Confidentiality of Their Responses. The first responsibility is related to user
input and states in part:
In addition, Federal statistical agencies and recognized statistical units must
seek input regularly from the broadest range of private- and public-sector data
users, including analysts and policy makers within Federal, State, local, tribal,
and territorial government agencies; academic researchers; and private sector
businesses and constituent groups.
The United States is not alone in identifying statistical principles. The European
Statistics Code of Practice adopted in September 2011 guides European statistical
systems to ensure high quality statistical production processes, to protect the
confidentiality of the information they collect, and to disseminate statistics in an objective
and transparent manner. This Code of Practice contains fifteen principles; Principle 11,
“European Statistics meet the needs of users,” deals with the topic of user input. In
1992, the United Nations (UN) Economic Commission for Europe (ECE) adopted a set of
“Fundamental Principles of Official Statistics” for the ECE region. When transition
economies began to move toward democratic governance, the UN Statistical
Commission adopted these principles in 1994. In 2011, the Commission reaffirmed the
ten principles, none of which explicitly mention users, but in 2013 revised the preamble
to note the importance of user participation and cooperation between users and
producers of statistics is to meet users' needs. The new preamble in part stated: “… the
quality of the information available to the Government, the economy, and the public
depends largely on the cooperation of citizens, enterprises, and other respondents in
providing appropriate and reliable data needed for necessary statistical compilations and
on the cooperation between users and producers of statistics in order to meet users'
needs.” On January 29, 2014, the UN General Assembly endorsed the principles, but
deleted this section from the preamble. As a result, none of the UN principles explicitly
mention users, as do the US and European principles.
3. Laws and Regulations
In most countries, including the US, the executive and legislative branches of
government make the final decisions on the scope of official statistics programs. In the
US, the executive is the President and the legislature (Congress) consists of the House
of Representatives and the Senate. Statistical programs are proposed by the President
and approved and funded by agreements between the two parts of the legislature. Thus,
US statistical programs are determined through the interactions of the President, the
Congress, the statistical agencies, and the respondent to and users of these programs.
3.1. Roles of the President and of the Congress
3
The President proposes statistical programs, and the Congress approves and funds
them, or rejects them. In addition, OMB, acting under the requirements the Paperwork
Reduction Act (PRA), sets regulations covering information collection and oversight of
statistical agencies. Under the PRA, OMB must approve the forms to be used to collect
the information for statistical and other federal programs.
3.12. The President and the Executive Branch
The President, after consultation with individual departments and agencies, submits to
Congress for its approval an annual budget that specifies the programs to be
implemented. In general, annual budgets proposed by the president have been
supportive of the goals of the statistical agencies and reflect their recommendations. In
past years, the President supported new programs, such as the American Community
Survey, the American Time Use Survey, the Longitudinal Employer Household
Dynamics program, and the Annual Capital Expenditures Survey. For the 2015 budget,
the President has proposed funding for new or expanded programs, including: Small
Business GDP, contingent work, research and testing of new methods for the 2020
decennial census, and research and development in the nonprofit sector. But even for
funded programs, statistical agencies cannot collect information to implement them
without approval by OMB. On rare occasions, the Executive Branch, including the
President, can issue “Executive Orders” telling statistical agencies to introduce new
statistical programs and to fund them out of their existing budgets.
3.13. Congress
The Congress approves or changes the President’s budget requests. The legislature
also can impose conditions on the frequency, content, survey methodology, and whether
respondents are required to participate in the collection of data. Recent actions by the
Congress have emphasized their role as they have tried to shape statistical programs
often without regard to the wishes of users. In some cases, users have been able to rally
support for particular programs and convince a majority of the legislators to reject the
changes. For example, Congress banned the use of sampling in the decennial census
and for the 2000 decennial census voted to include questions on grandparents who were
primary care givers for children. In 2012, one part of the legislature voted to eliminate the
Census Bureau’s quinquennial economic census; that effort was defeated after strong
user objections. In 2014, a proposed improvement in the wording of questions on a
household survey on health-insurance coverage was blocked because too many
members of Congress thought the changes were politically motivated. On survey design,
one part of the Congress has voted to convert the ACS from a mandatory to a voluntary
survey but not to increase the funding needed to maintain quality if that change were
made. This year, the Congress considered requiring the Census Bureau to include
questions on immigration and citizenship in the 2020 decennial census, but that effort
was defeated. In some of these instances, the Congress was acting because of
complaints by respondents, whether businesses or persons. The current discussion to
make the ACS a voluntary survey was prompted largely by what households saw as an
overly intrusive survey, which may indicate a lack of trust in government agencies.
3.14. User Input to the Executive Branch and Congress
4
Users provide input to the Executive Branch on statistical programs in three ways. As
individuals, users can contact the President or Department or agency heads with their
comments. They also can serve as members of official “advisory” committees to specific
agencies or submit comments to OMB as part of the information collection process.
They also can have organizations to which they belong provide comments. It should be
noted that individuals and organizations submitting comments could either support or
oppose changes to statistical programs.
User input to Congress is more limited primarily because the organizations that support
federal statistics are incorporated as nonprofit institutions and are barred from
attempting to influence members of Congress.
4. Legislation – The Paperwork Reduction Act
In the US, legislation and related regulations have formed the basis for users and
respondents to provide input into statistical programs. The major law that currently
guides US statistical agencies, as well as most other agencies, to obtain user input is the
Paperwork Reduction Act (PRA), which specifies what agencies must do obtain approval
to collect data. This section describes the history of the provisions of this act and its
related regulations and comments on its effectiveness.
In 1933, the President created a Central Statistical Board to advise him on the federal
government’s collection of statistics required to carry out the National Industrial
Recovery Act, to review design of tabulations, and to promote the coordination and
improvement of these statistics. In 1939, these functions were transferred to the Bureau
of the Budget (BOB), the predecessor agency to OMB. In 1942, the Federal Reports Act
gave BOB authority to coordinate and oversee federal statistical agencies. This Act also
banned federal agencies from collecting data from 10 or more respondents without
approval. In 1950, other legislation further strengthened BOB’s role by giving it authority
to issue regulations and orders governing all federal statistical programs.
This information collection and statistical policy function continued when the BOB
became the OMB in 1970. Despite OMB’s oversight activities of information collection,
the Congress was displeased with the amount of data being collected and its burden on
respondents. This displeasure led to the passage in 1980 of the first PRA, which was
amended in 1986 and most recently in 1995. The original PRA strengthened OMB’s
review power throughout the federal government over the collection of information,
including agency budgets for statistics as well as the methods used for data collection
and dissemination. According to OMB, the PRA was designed, among other things, to “
… ensure the greatest possible public benefit from and maximize the utility of
information created, collected, maintained, used, shared and disseminated by or for the
Federal Government and to improve the quality and use of Federal information to
strengthen decision making, accountability, and openness in Government and society.
Most importantly, as the title PRA implies, OMB was to reduce the burden imposed by
regulatory as well as by administrative and statistical program agencies. 1 The PRA
required agencies to undertake efforts to reduce the burden of information collections
and to obtain formal input into the scope and the content of the collections. The PRA
also required OMB, through its Chief Statistician, to improve federal statistical programs;
1
According to the Government Accountability Office, only a small percentage of the burden is
from information collections for statistical purposes; about 80 percent are for tax returns.
5
review statistical budgets; coordinate government statistical functions; establish
standards, classifications, and other guidelines for statistical data collection and
dissemination; and evaluate statistical program performance. The 1995 Act added
provisions on statistical policy to formalize (1) the Interagency Council on Statistical
Policy (ICSP) and (2) OMB’s role in coordinating US participation in international
statistical activities.
The current PRA and related regulations spell out the OMB survey clearance process,
which requires Federal agencies to seek public comment on proposed collections and to
submit proposed collections for review and approval by OMB. (Agencies are not required
to report to OMB the elimination of a collection.) OMB either approves the request with
or without revisions or disapproves the request. To obtain the public’s input on an
agency’s proposal to collect information, the PRA generally requires the agency to
publish a 60-day notice in the Federal Register. 2 The notice must include a specific
request that the public comment on whether the proposed collection is necessary; on the
accuracy of the agency’s estimate of respondent burden; on how to enhance the quality,
utility, and clarity of the information to be collected; and on how to minimize the burden
of the collection of information.
To publish a 60-day notice, the PRA requires agencies to provide OMB a draft
information collection plan, the survey instrument, and the following information about
the survey:

It is necessary for the proper performance of agency functions;

It avoids unnecessary duplication and reduces burden on small entities;

The survey document informs respondents as to why the information is being
collected and the use of information;

Whether response is voluntary, or mandatory and nature and extent of
confidentiality protection;

The degree of compatibility with existing reporting and recordkeeping
practices of those who are to respond and an estimate of the average burden
of the collection; and

The use of effective and efficient statistical survey methodology.
At the end of the 60-day comment period and the agency’s consideration of the public’s
comments, the agency submits the collection to OMB for approval and publishes a
second Federal Register notice to announce the start of the OMB review. This second
notice informs the public about how to submit additional comments to OMB and informs
the public that OMB may act on the agency’s request only after the 30-day comment
period has closed. This 30-day notice should include a summary of public comments
received in response to the first notice and describe actions taken by the agency in
response to these comments and describe efforts to consult with persons outside the
agency. Agencies also are required to consult with respondents at least once every 3
years. OMB will decide whether to approve or disapprove or to instruct the agency to
make specific changes to its proposed collection. The results of the OMB review are not
published in the Federal Register, but are posted on a special OMB website,
http:www.reginfo.gov/public/do/PRASearch.
2
The Federal Register is a daily report that contains the official record of notices, regulations, and
other documents released by federal government agencies.
6
To summarize, the usual survey clearance process consists of the following five steps:
(1) Agency develops information request, (2) 60-day Federal Register notice is
published, (3) Agency considers public comments, (4) 30-day Federal Register notice is
published announcing that the agency has submitted its request for OMB review, and (5)
OMB approves as submitted, approves with revisions, or disapproves.
5. User Input
User input to the President, the Congress, and the statistical agencies comes from
several major sources: The PRA survey clearance process; direct outreach efforts by the
agencies, including evaluation by the National Academy of Sciences; other federal
government agencies; and independent groups of users, including professional
organizations and associations. The previous section discussed input under the
clearance process; the following sections of the paper will provide an overview from
other major sources.
5.1. Direct Agency Outreach
Statistical agencies solicit input from users using formal advisory committees,
conferences, participation in the activities of professional organizations and associations,
Internet announcements and webinars, and contracts with outside organizations. In
some cases, these approaches are co-sponsored by professional organizations and
associations.
5.11. Advisory Committees
Agencies are allowed by legislation to establish advisory committees to provide
recommendations on agency programs. Members of these committees are selected for
their expertise in fields such as economics, statistics, and demography; they may work
for businesses, nonprofit organizations, and other federal and local government
organizations. Meetings of these committees are open to the public, and a period must
set aside for public comments and questions. The format for these meetings varies from
agency presentations and questions to presentations by the members. For example,
these committees have been established by the Bureau of Economic Analysis (BEA),
the Bureau of Labor Statistics (BLS), the Census Bureau, and the National Agricultural
Statistics Service. In addition, the Federal Economic Statistics Advisory Committee
advises the Directors of the Economics and Statistics Administration’s two statistical
agencies, the BEA and the Census Bureau, and the Commissioner of BLS on statistical
methodology and other technical matters related to the collection, tabulation, and
analysis of economic statistics.
5.12. Input from Other Statistical Agencies
The Chief Statistician of OMB directs two major efforts to provide input to the programs
of statistical agencies. The Interagency Council on Statistical Policy (ICSP), formally
established by provisions on the 1995 PRA, and the Federal Committee on Statistical
Methodology (FCSM), established by OMB in 1975.
The ICSP provides OMB with direct input by the heads of principal statistical agencies in
planning and coordinating Federal statistical activities. Some of its current activities
include the Interagency Committee for ACS, which is discussed later in this paper; the
7
Interagency Council on Agricultural and Rural Statistics, whose members are experts
from economic, demographic, environmental and agricultural agencies that maintain
statistical programs and from natural resource agencies to improve the coordination and
production of national statistics related to agriculture, food, natural resource and rural
data; and the Interagency Forum on Aging-Related Statistics, which support
collaboration among Federal agencies that produce or use statistical data on the older
population. An annual report on the activities of the ICSP is included in OMB’s Statistical
Programs of the United States, a report to Congress required under the 1995 PRA.
The FCSM advises OMB and the ICSP on methodological and statistical issues that
affect the quality of federal data. The FCSM prepares reports and provides
recommendations on issues of statistical methodology such as survey methods, survey
errors, data collection methods, and disclosure. (It has published over 30 reports and
working papers.) It also provides a forum for statisticians in different federal agencies to
discuss issues affecting federal statistical programs, and supports cooperative research
across agencies on issues relevant to federal statistics. FCSM members are career
Federal employees selected by OMB based on their individual expertise and interest in
statistical methods. Members serve in their personal capacity rather than as agency
representatives, and remain as members as long as they remain employed in an agency
of the Federal government covered by the Paperwork Reduction Act (PRA). The FCSM
also co-sponsors an annual Statistical Policy Seminar to present papers on the latest
developments; proceedings of these seminars are available at the FCSM web site.
5.13. Input from Contracts with Outside Organizations
Statistical agencies also solicit user input through contracts primarily with the Committee
on National Statistics (CNSTAT), a committee of the National Academy of Sciences
(NAS) established in 1972 to improve statistical methods and information from statistical
agencies. CNSTAT primarily accomplishes its mission by organizing panels of experts to
conduct studies, usually under contract with an agency or at the request of Congress.
The panels of experts serve on a voluntary basis with assistance of a small professional
staff. In addition to the preparation of Principles and Practices for a Federal Statistical
Agency, CNSTAT has recently published reports on such statistical programs as the
Census Bureau’s decennial census and ACS and the BLS’s Consumer Expenditure
Survey and American Time Use Survey. It also prepared a report on household survey
nonresponse. A complete list of CNSTAT reports can be found on its web site.
5.2. Direct User Generated Input
Users provide input into statistical programs not only through the PRA survey clearance
process but also through professional associations representing private and
research/academic communities such as the National Association for Business
Economics (NABE), the Council of Professional Association (COPAFS), and the
Association of Professional Data Users (APDU). These groups hold regular userproducer meetings, work with the statistical agencies to conduct Webinars on specific
programs and, most important, work to educate legislatures on the critical value of
statistical programs threated with elimination. Through these activities, these groups
provide their members with the opportunity to have input to statistical agencies on
existing and new programs, to influence the Executive Branch to support these
positions, and to publicly express their needs.
8
5.2.1. National Association for Business Economics
The National Association for Business Economics (NABE) represents more than 2,500
members; most are economists who work at the countries leading companies.
Representing the largest group of major users of federal economic and related statistics,
NABE maintains regular contact with statistical agencies to maintain and improve
statistical programs essential to its members primarily through two activities – a
Statistics Committee, and an annual Economic Measurement Seminar (EMS).
The NABE Statistics Committee, which meets quarterly, features regular reports by the
heads of the major statistical agencies on key economic statistics programs. They
provide the Committee members with the opportunity to provide input directly to the
agency heads and provide agency heads the opportunity to solicit input on their
programs from major users. When informed that funding for the 2012 Economic Census
and virtually all Census Bureau current programs was in jeopardy, NABE worked with
other professional associations to sponsor educational briefings for Congress on the
need for these programs. In both of these examples, these proposals were ultimately
defeated in Congress.
The annual EMS is a unique two-day program, now in its 11thyear, that consists of
sessions on key statistical programs with a representative of the data producer and a
prominent NABE data user. The several hundred attendees learn how the agencies
compile the data and how it is used leading business analysts.
5.2.2. Council of Professional Associations for Federal Statistics
The Council of Professional Associations on Federal Statistics (COPAFS), founded in
1981, includes as its members about 50 professional associations representing disciples
such as economists, statisticians, psychologists, and sociologists and associations such
as NABE, American Economics Association, American Statistical Association,
Population Association of America, Association for Public Opinion Research, Association
for University Business and Economic Research, and the National Association for Public
Health Statistics and Information Systems. COPAFS members also represent private
data collecting and analyzing organizations such as Westat, National Opinion Research
Center, Sabre Systems, and RTI international. In all, COPAFS represents over 300,000
individual academics and businesses researchers, as well as local governments.
COPAFS provides its member organizations with opportunities to provide input to policy
makers. It holds regular quarterly meetings, as well as forums, seminars, workshops,
and webinars, to keep its members up-to-date on developments that impact their use of
federal statistics. COPAFS also partners with similar private organizations to educate the
Congress about the importance of funding various statistical programs.
For example, COPAFS participated with NABE and other groups in 2013 in a major
educational effort to defeat proposed legislation that would eliminate all Census Bureau
surveys, except the decennial census of population, COPAFS mobilized its member
organizations to tell Congress of the disastrous impact on the nation if this legislation
were enacted. It was not enacted.
Most recently, COPAFS led the effort to form a group called the “Friends of the Bureau
of Labor Statistics (BLS).” This group was formed when earlier this year BLS in
9
response to significantly reduced funding, decided to end its collection of export prices, a
key economic indicator and a major source of data needed to prepare national accounts
estimates. As one of the “friends,” NABE wrote to the Congress to reverse the funding
reductions. Although Congress did not restore the cuts, other agencies decided to
support the program for another year. The main objectives of the “Friends of BLS” are to
create a network of BLS data users, to distribute information and ideas about BLS
programs, and to identify educational events for the public and public policy decision
makers that support BLS programs. “Friends” now consists of over 30 organizations,
four businesses, and 15 researchers.
5.2.3. Association of Public Data Users
Members of the Association of Public Data Users (APDU) consists of users, producers,
and disseminators of government statistical data, including professional associations,
state and local government agencies, universities, and other users of data on small
geographic areas. APDU represents these public data users on important issues of
government information and statistical policy and provides its members a means to
share information and concerns.
Webinars are a major part of APDU’s training and professional development program.
APDU has partnered with statistical agencies to develop a series of about 60 individual
webinars throughout 2014 and into 2015. In 2014, APDU conducted webinars on both
demographic and economic statistics covering the products of almost all of the principle
statistical agencies. These webinars are broadcast live and are recorded for later use. In
addition, representatives from APDU regularly meet with members, partners, and
policymakers to keep up to date on issues that affect public data. APDU also represents
its members on organizations including Friends of BLS, the Decennial Census Advisory
Committee, and COPAFS.
In addition, APDU prepares a weekly newsletter that keeps members up-to-date on the
activities of statistical agencies such as new publications, advisory committee meetings,
and webinars. The newsletters also inform members of new and revised information
collections listed in the Federal Register, as required by the PRA, that are open for
public comment. As noted previously, the comment process provides APDU members
the opportunity to provide input on the value, content, and methods used to collect
federal data. APDU also holds a 2-day annual conference to provide members another
opportunity to communicate with data producers and to discuss latest developments.
6. Examples of User-Producer Interactions
This section provides an overview of how user input was solicited for two major
statistical programs. The two examples are the American Community Survey (ACS), a
relative new survey of the Census Bureau and the Consumer Expenditures Survey (CE),
a long-standing program of the BLS.
6.1. American Community Survey
The American Community Survey (ACS) is a relatively new survey conducted as part of
the decennial census of population. It replaced the “long form” sample component of the
census, previously collected at the same time as the basic demographic information
covered by the census. The ACS uses monthly samples and the continuous
10
measurement methodology to produce annually updated estimates of detailed
socioeconomic information for the same small areas (census tracts and block groups).
To produce these small area data, five years of samples are required. The Bureau
released its first 5-year estimates in December 2010; new small-area statistics now are
produced annually using successive 5-year periods. The Census Bureau also produces
3-year and 1-year estimates for larger geographic areas.
The Bureau began to consider continuous measurement as a possible alternative to the
long form in the mid 1990s to meet policymaker’s demands for more current and
nationally consistent small area data. In addition, the Bureau saw possible cost savings
and improved population counts id the long form were removed from the decennial
census. The Bureau developed a plan to implement continuous measurement in 2000,
which later became known as the ACS. During the development phase of the program,
the Bureau consulted with users including federal, state, and local governments,
advisory committees, and professional organizations. The goal was to get feedback on
the overall approach and to identify potential pitfalls and obstacles. The Bureau
contracted with CNSTAT to identify key issues with the design of the survey and with a
private organization, Westat, to organize a discussion issues relating to areas with small
populations. In addition, presentations were made to the Congress. Data users and the
public were invited to attend community workshops to learn about the ACS. From March
1996 to November 1999, there were 31 town hall-style meetings and, in mid-2004, an
additional three regional outreach meetings. Also, because use of long-form data had
been required for numerous government programs, ACS staff began a series of
information meetings with the key federal agencies. In 2013, the Bureau contracted with
CNSTAT to review, assess, and provide recommendations on research to improve ACS
estimates of small populations.
By 2004, the Bureau began to focus on ACS data products and, as required by the PRA,
requested public comments in a Federal Register notice. Since then, the Bureau has
continued to solicit user input through the ACS Alert, a periodic Bureau electronic
newsletter. In 2013, the Bureau, in partnership with the Population Reference Bureau
and Sabre Systems, formed an ACS Data Users Group to improve user understanding
of the ACS and discussions among ACS users. Since its formation, the Group has
sponsored an ACS Data Users Conference (May 2014), webinars, and professional
association conference presentations. In March 2014, the Bureau formed an ACS
Content Review process to discuss with federal agency users whether the ACS
continues to meet their needs.
Perhaps the most unusual part of the ACS outreach effort was the publication from 2008
to 2010 of a set of 12 educational Compass Product Handbooks. These reports
recognized the complexities of ACS data and the varied needs of users. The Bureau
addressed this diversity by working closely with groups of experts outside the federal
government to develop the handbooks, each of which is designed for a particular
audience. Although these handbooks do not cover every situation, they provided an
important educational process for ACS users.
6.2. Consumer Expenditure Survey
The Consumer Expenditure Survey (CE) program provides information on all types of
consumer expenditures and incomes, as well as characteristics of consumers. The CE is
used to monitor the impact of policy changes on economic groups, by the Census
11
Bureau as input for calculating thresholds for measures of poverty, by researchers
studying trends in consumer spending, and by the BLS to update the market basket and
weights of goods and services for the Consumer Price Index (CPI).
6.21. Gemini Project
In February 2009, BLS initiated a multiyear survey redesign effort, the “Gemini Project,”
to improve CE expenditure estimates by reducing measurement error and increasing
response rates, but without increasing funding or reporting burden. A redesigned
methodology was selected in 2013 and an extensive testing program will continue until
2018. Revised questions will be determined in 2015 and also incorporated into the
testing program.
To obtain user input on both the survey content and methodology, BLS sponsored a
series of information-gathering meetings, conference sessions, forums, and workshops
to identify ways of improving CE data quality. Meetings were held in 2009 and 2010,
followed by analysis and research in 2011. Also in 2011, BLS contracted with CNSTAT
to conduct further investigations and propose redesign options. In addition, BLS made
presentations for users at the National Bureau of Economic Research’s Conference on
Improving Consumption Measurement (July 2009); and at a panel discussion on survey
redesign cosponsored by the Washington Chapter of the AAPOR and the Washington
Statistical Society (January 2010). In July 2014, BLS held a survey methods symposium
to discuss past and future developments.
At the same time, a BLS staff team, called the Data User Team, held an event in June
2010 where attendees reported on their uses of the CE data, data needs and priorities,
and recommendations for changes. In December 2010, they held a Survey Redesign
Workshop where private sector firms presented various technologies used to collect
survey data. A Conference Team organized a panel at the 2010 American Association
for Public Opinion Research meeting. A report summarizing the 2010 event and similar
input from the CPI staff was used to create a list of CE priorities as well as other data
requirements. This report served as a key reference for all subsequent discussions
regarding potential design alternatives. Also in 2010, a Steering Team held several
major events and conducted additional research studies, including examining methods
used in international consumer expenditure surveys.
6.23. CNSTAT Contract on the CE
The Gemini project also included a contract with the CNSTAT for an expert panel to
build upon the Gemini Project information gathering by conducting further investigations
and proposing redesign options for the CE surveys. The charge to the “Panel on
Redesigning the BLS Consumer Expenditure Surveys” included reviewing the output of
the BLS meetings on user data needs and survey methods, as well as holding a data
user needs forum and a methods workshop. CNSTAT was also to conduct a Household
Survey Producers workshop to learn how leading survey organizations deal with the
types of challenges faced by the CE, and a workshop on redesign options with papers
prepared under contract by CNSTAT.
The panel held four meetings and two workshops before preparing its report. The
Household Survey Producers Workshop was held in June 2011. The panel, which
included several members who were CE users, received input from a wide variety of
12
sources, with investigations conducted by the Gemini Project providing critical
background. The panel also reviewed published research and held a session at the 2011
CE Microdata Users’ Conference. The final CNSTAT report, Measuring What We Spend,
summarized the activities of the panel. It provided conclusions about the uses of the CE,
why a redesign was needed, as well as recommendations for three prototype designs. It
recommended that the first step should be for BLS to determine priorities among the
data requirements.
7. User Input in US -- Conclusions
This paper reviews the opportunities in the US for users of federal statistics to provide
input directly to the agencies that produce them. These opportunities result from longstanding legislation and regulations related to these statistics, the desire by statistical
agencies to follow best practices, which include soliciting user input, and the widespread
reliance on government statistics by government, business, and individuals.
Nevertheless, with a few exceptions, such as the introduction of the new ACS, it is hard
to find concrete examples where agencies have accepted user input. On the other hand,
agencies are always willing to partner with users to prevent the Congress from cutting
funding or eliminating programs.
This lack of users’ general success can be attributed to factors such as agency
resistance to suggestions for changes from outsiders, conflicting priorities among users,
difficulty in obtaining OMB approval because of legitimate respondent objections on
burden or intrusive questions, survey cost, and collection of information that cannot be
reliably reported. Agency resistance to change is particularly evident when agencies
contracting for CNSTAT studies do not respond to their recommendations.
Despite this apparent lack of success, the extent of user input has had a very beneficial
impact on both users and producers. The continuous flow of input has resulted in
ongoing communication between the two groups that has resulted in education about
how the data are used and how the data are collected. It is likely that in the long run,
agencies do take into account user input in their program design. Unfortunately, actions
by the Congress on statistical agencies frequently ignore the users of the data, and
some times reflect a political agenda.
References:
Association of Public Data Users web site at www.apdu.org.
Bureau of Labor Statistics, “Gemini Materials,” web site at
www.bls.gov/cex/geminimaterials.htm.
Council of Professional Associations for Federal Statistics web site at www.copafs.org.
Eurostat, European Statistics
epp.eurostat.ec.europa.eu.
Code
of
Practice,
September
28,
Government Accountability Office, New Approach May be Needed to Reduce
Government Burden on Public, 2005, Report GAO-05-424 at www.gao.gov.
13
2011,
at
Government Accountability Office testimony before Congress on the Paperwork
Reduction Act report GAO-04-676T at www.gao.gov.
Government Accountability Office, “Key Requirements of the Paperwork Reduction Act
and OIRA Actions,” Appendix II, Information Resource Management: Comprehensive
Strategic Plan Needed to Address Mounting Challenges, 2002, Report GAO 02-292 at
www.gao.gov.
Government Printing Office, Federal Register, May 21, 2014, “Fundamental
Responsibilities of Federal Statistical Agencies and Recognized Statistical Units.” at
www.GPOaccess.gov.
Government Printing Office, Federal Register, August 29, 1995, “Controlling Paperwork
Burdens on the Public; Regulatory Changes Reflecting Recodification of the Paperwork
Reduction Act,” at www.GPOaccess.gov.
National Academies of Sciences, Principles And Practices For A Federal Statistical
Agency, Fifth Edition, at www.national academies.org/cnstat.
National Academy of Sciences, “Survey Clearance Process,” Appendix A, Principles And
Practices For A Federal Statistical Agency, Fifth Edition, at www.national
academies.org/cnstat.
National Academy of Sciences, Measuring What We Spend: Toward a New Consumer
Expenditure Survey, 2013. Workshop Discussion of NRC Report, at www.national
academies.org/cnstat.
National Association for Business Economics web site at www.nabe.com.
Office of Management and Budget, “OMB Statistical
http://www.whitehouse.gov/omb/inforeg_statpolicy.
Policy
Directives,”
at
Office of Management and Budget, Statistical Programs of the United States: Fiscal
Year 2014, at http://www.whitehouse.gov/omb/inforeg_statpolicy.
Office of Management and Budget, “Strengthening Federal Statistics,” Budget of the
United States, 2015, at http://www.whitehouse.gov/omb/inforeg_statpolicy.
Office of Management and Budget, “Information Collection under the Paperwork
Reduction Act,” OMB memorandum, April 7, 2010, OMB web site.
http://www.whitehouse.gov/omb/inforeg_statpolicy.
United Nations Statistical Commission, Fundamental Principles of Official Statistics,
2014, at http://unstats.un.org/unsd/dnss/gp/fundprinciples.aspx.
U.S.
Census
Bureau,
www.census.gov/acs/www.
American
Community
Survey
web
site
at
U.S. Census Bureau, A Compass for Understanding and Using American Community
Survey Data: What General Data Users Need to Know, 2008, at
www.census.gov/acs/www.
14