Formulas, Ratios, Estimates, and
Counts…
The Historical Roots of Quantitative
Public Policy in the U.S
Margo Anderson
History & Urban Studies
University of Wisconsin – Milwaukee
margo@uwm.edu
Outline and Themes
• The federal statistical system today is wrestling with
challenges to produce timely, accurate, efficient, and
intelligible demographic and economic data for public
policy.
• It’s an old problem.
• A brief historical survey reminds us that deploying
quantitative metrics for public decision making dates
from the very beginning of the republic and shapes the
world we live in today.
2
Federal Statistical System Today
• Decentralized: Census, BLS, NASS, NCHS,
NCES, BJS, etc.
• 98 agencies with statistical activities; 13
lead agencies
• Federalized: states and local governments
also provide data through coordinating
arrangements: e.g., vital statistics
• Partnerships: with research organizations
and universities to collect and produce data
products
3
The Statistical System within the
Federal Government
• Chief Statistician resides in OMB and
coordinates the system through “forms
clearance” and budget authorizations.
• Legislative grounding is in the Paperwork
Reduction Act of 1995.
• A diverse system.
• A public system, grounded in the political
and policy making activities of the nation.
4
Federal Statistical System
• Overall budget of about $6.8 billion/year (exclusive
of decennial census).
• About 40% of expenditures in 13 lead agencies
• Current challenges:
–
–
–
–
–
–
–
Respondent cooperation.
Fiscal resource constraints.
International comparability.
More detailed data
Access to data
Statistical agency independence
Human capital – next generation…
5
Budgets: Large Agencies (millions of $)
6
Origins and Structure
• History helps!
– The 1787 Constitution created the platform of the
system.
– The Constitution created two different types of
statistical or public data collection and reporting:
• the decennial census
• the reports on government revenue and
expenditures
7
Understanding the Statistical
System
• As embedded in the larger political, social,
economic and demographic situation of the
US
• As shaping the larger political, social,
economic and demographic situation of the
US
• As a technical system
8
Useful Distinctions
• Survey Data: Data collected for research or
policy purposes only, usually sampled: CPS,
SIPP, ACS
• Administrative Data: Data collected for
administrative functions and then reused or
reorganized for statistical data analysis: state
unemployment records; tax records, property
records, medical records.
9
Institutional and Some Technical
History
• “Constituting” the system: 1780s
• Implementing the system in the long 19th
century, 1790- early 1900s.
• The centralization/coordination debate, 19001940s
• The modern system
10
Building the American State
11
12
Article 1, Section 2, of the U.S.
Constitution
• "Representatives and direct Taxes shall be
apportioned among the several States which may
be included within this Union, according to their
respective Numbers….The actual Enumeration
shall be made within three Years after the first
Meeting of the Congress of the United States, and
within every subsequent Term of ten Years, in
such Manner as they shall by Law direct."
13
Importance of the Census
• The United States was the first
nation in the history of the world to
take a population census and use it
to allocate seats in a national
assembly according to population.
14
Implementing the System
• First census was taken in 1790.
• The House of Representatives and Electoral
College was first reapportioned in 1792
• Immediately, government officials and the
general public recognized the significance of
the new system for allocating representation.
15
Pitcher Commemorating the 1790
Census
16
Census
Publications
17
Francis Edmonds, Taking the Census,
1853
18
“The Great
Tribulation,”
The Saturday
Evening Post,
1860
19
Importance of the Census
• The U.S. has had one of the most demographically
dynamic and diverse populations in the history of the
world.
• The combination of the census as mechanism to
adjust power and resources each decade, in
conjunction with the demographic dynamism and
diversity, made the census and the statistical system
truly central to the functioning of the society and
state
20
Importance of the Census
• Dynamism is measured by patterns of population
growth and change
• Diversity involves geographic diversity, group
diversity, and different rates of change for different
parts of the country, and among the groups.
• Hence three levels
– Numerical growth
– Geographic diversity
– Racial and ethnic diversity
21
Numerical Growth
From 3.9 million to 314 million
• 13 states have become 50 states.
• House of Representatives grew from 65 to 435
members.
• The average congressional district today is larger
than the total population of any of the original 13
states in 1790.
• Growth has been differential: some states and local
areas lose while others gain.
23
Population Growth, 1790-2000
300
Population (millions)
250
200
UK
FR
150
US
100
50
0
Year
Admitting States to the Union and
Growing the House of Representatives
25
Differential Population Growth: New
York State Population and House
Delegation, 1790-2010
26
How Is Growth Managed?
• Decisions to be made on….
– Size of the House of Representatives
– Admission of new states to the union
– Rules for creating congressional districts
• Apportionment Formulas:
–
–
–
–
–
Hamilton’s Method
Jefferson’s Method
Vinton’s Method
Webster’s Method (major fractions)
Hill’s Method (equal proportions)
Geographic Diversity
Geographic Diversity:
Westward Expansion
29
Geographic Diversity:
The First Gerrymander, 1812
30
31
Racial and Ethnic Diversity
Article 1, Section 2, Paragraph 3
• Representatives and direct Taxes shall be apportioned
among the several States which may be included
within this Union, according to their respective
Numbers, which shall be determined by adding to the
whole Number of free Persons, including those bound
to Service for a Term of Years, and excluding Indians
not taxed, three fifths of all other Persons. The actual
Enumeration shall be made within three Years after
the first Meeting of the Congress of the United States,
and within every subsequent Term of ten Years, in
such Manner as they shall by Law direct.
Why were these provisions put in the
Constitution?
• The framers had to decide the basis for
allocating:
– representation; and
– revenue obligations among the states
• The rule:
– revenue should be apportioned according to
ability to pay or wealth;
– representation should be apportioned according
to the number of people in the state
Federal Ratio or Three Fifths Rule
Undoing the Original Constitutional
Framework
• Abolishing Slavery: 13th Amendment
• Amending the Constitution: Wartime
Amendments: 14th and 15th Amendments
• Breaking the Link between Taxation and
Representation: 16th Amendment
Constitutional Revisions
• The Thirteenth Amendment (1865) abolished
slavery and the ‘federal ratio.’ The
amendment ended the discounting of the
formerly slave population in the allocation
formula.
• Abolition increased the level of representation
of the former slave states in the House of
Representatives and Electoral College and led
to further constitutional revisions.
Constitutional Revisions, New Ratios
• Fourteenth Amendment, Section 2 (1868):
– “Representatives shall be apportioned among the several
States according to their respective numbers, counting the
whole number of persons in each State, excluding Indians
not taxed. But when the right to vote at any election for the
choice of electors for President and Vice-President of the
United States, Representatives in Congress, the Executive
and Judicial officers of a State, or the members of the
Legislature thereof, is denied to any of the male inhabitants
of such State, being twenty-one years of age, and citizens of
the United States, or in any way abridged, except for
participation in rebellion, or other crime, the basis of
representation therein shall be reduced in the proportion
which the number of such male citizens shall bear to the
whole number of male citizens twenty-one years of age in
such State.”
Constitutional Revisions
• Fifteenth Amendment (1870): “The right of
citizens of the United States to vote shall not be
denied or abridged by the United States or by any
State on account of race, color, or previous
condition of servitude.”
• Sixteenth Amendment (1913): “The Congress
shall have power to lay and collect taxes on
incomes, from whatever source derived, without
apportionment among the several States, and
without regard to any census or enumeration.”
First Reading of the Emancipation
Proclamation
40
Civil War Demographic Map
Administrative History of the Census,
1790-1902
• From 1790 to 1902, a temporary agency in the
Department of State or Interior.
• Until 1880 the US marshals and their assistants
served as the field staff.
• Over the years, Congress added the collection of
agricultural, manufacturing, mortality, disability
statistics to the decennial.
• A very large administrative operation during the
census period, but administrative discontinuity.
• Congress considered proposals for a permanent
census office but did not act on them until 1902.
42
Who Decides?
• Congress!
Meanwhile….
• The other constitutionally mandated “leg” of
the system developed.
44
Economic and Administrative Statistics
• Article 1, Section 9: “a regular Statement and
Account of the Receipts and Expenditures of all
public Money shall be published from time to time.”
• Article 2, Section 3: The President “shall from time to
time give to the Congress Information of the State of
the Union and recommend to their Consideration
such Measures as he shall judge necessary and
expedient.”
45
Implications….
• Administrative records of the revenue and
expenditure of government were collected
and published, making it feasible to develop
administrative statistics.
• The government created an administrative
structure to collect, analyze and publish the
data.
46
Private Publications (Partnering) of
Federal Statistics Begin Very Early!
• Timothy Pitkin, A Statistical View of the Commerce of
the United States of America (1816)
• Adam Seybert, Statistical Annals: Embracing Views of
the Population, Commerce, Navigation, Fisheries,
Public Lands, Post-Office Establishment, Revenues,
Mint, Military and Naval Establishments,
Expenditures, Public Debt and Sinking Fund, of the
United States of America, Founded on Official
Documents, 1789-1818
47
Routine Statistical Reporting Started
in the Treasury Department
• 1820: The Secretary of the Treasury began
to prepare annual statistical accounts of the
commerce of the US with foreign countries.
• 1840-1860s: Congress authorized hiring of
clerks, regular publication of reports.
48
The Treasury Department and
Permanent Statistical Offices
• 1866, Bureau of Statistics established in the
Treasury Department.
• 1878, the Bureau of Statistics published the
first edition of the Statistical Abstract of the
United States.
49
Statistical Agencies Established
in Other Departments
•
•
•
•
Agriculture Department: 1862
Bureau of Education: 1867
Bureau of Labor: 1884
Immigration Statistics: collected in the
Treasury Department and State
Department
50
At the Dawn of the Twentieth
Century…
• The United States was recognized as a pioneer in
statistical methodology and technology: machine
tabulation of the census began in 1890.
• Congress had been successfully reapportioned 12
times, 46 states were in the union, and legislatures
had learned to redistrict on the basis of geographic
growth and change.
• Routine, reliable data poured out from federal
statistical offices, guided policy development on the
tariff and taxation, immigration policy, disability,
labor relations, and many more areas.
The World of
Printed Reports:
Statistical
Abstract, 1902,
580 pages
52
53
54
New Challenges to the Statistical
System
•
•
•
•
•
Industrial Revolution
Rapid City Growth
Growth of the Working Classes
Disparities between Rich and Poor
Problems of Boom and Bust: Panics,
Depressions and Crashes: 1857; 1873-77;
1893-97; 1907; 1921
Examples of the Development of New
Data Series…
• Price, Expenditure, and Cost of Living
Measurement
• Measuring Unemployment and the Business
Cycle
56
Cost of Living
Measurement
57
58
Followed by Cost of Living Survey Series
• The Cost of Living Survey series was created
by the United States Department of Labor,
Bureau of Labor Statistics (BLS), with the goal
of estimating the cost of living of a "typical"
American family.
– COST OF LIVING OF INDUSTRIAL WORKERS IN THE
UNITED STATES AND EUROPE, 1888-1890 (ICPSR
7711),
– COST OF LIVING IN THE UNITED STATES, 19171919 (ICPSR 8299)
59
Characteristics of the Tradition
•
•
•
•
•
Household based survey
Relatively small number of cases
Huge number of variables
Sampling structure is complex
Conducted relatively infrequently
Measuring the Business Cycle and
Unemployment
• Business cycles are irregular and frequent
• They are characterized by seasonality as well
as periodicity
• They call for different measurement strategies
• And raise questions: is the unit of analysis:
– The firm?
– The household?
– The overall economy?
Strategies….for Unemployment
Measurement
• Administrative Data:
– Labor Exchanges
– Industry Reports of Activity
• Census, 1880-1910; 1930
– Problems:
• Infrequent data
• Conceptual definitions
Mary Van Kleeck
on Measuring
Unemployment,
1923
Business Cycles and Unemployment….
• “If the facts [data on employment and unemployment]
are to be useful …they must be widely enough scattered
geographically not to be over-influenced by condition
which may be merely local in one section of the country;
they must be made available by some central agency
which can correlate and interpret them; and, perhaps
most important of all, they must be made public with
sufficient promptness to be approximately true
measures of the state of employment at the time when
they are issued. Thus the problem of extending and
improving employment statistics is less statistical in its
nature than it is administrative. It demands a machinery
strong enough and simple enough to work smoothly and
rapidly without breakdowns.”
Was the Statistical System up to the
Task?
• The statistical system in the early 20th century
was somewhat anarchic and duplicative.
• Many of the thornier issues of the
Constitutional era continued to plague the
statistical system, what might be called a
“politics of population.”
• In 1903, with the Organic Act creating the
Department Commerce and Labor, Congress
mandated consolidation and coordination.
• It failed.
67
Coordination and the Politics of
Population
• Theodore Roosevelt’s problem: merging agencies
with traditions of administrative data and survey
data.
• Herbert Hoover’s problem: the reapportionment
battles of the 1920s, the only time in the history of
the republic that Congress refused to reallocate
House seats among the states on the basis of the
census results.
• Franklin Roosevelt’s problem: the Great Depression
and measuring unemployment.
Trying Again…and Learning to Live with
Decentralization
• Bureau of Efficiency, 1920s
• COGSIS, Committee on Government Statistics
and Information Services, 1930s
• 1940: Position of Chief Statistician created in
the consolidation of the Bureau of the Budget
(now OMB)
• 1942: Federal Reports Act mandated “forms
clearance.”
69
Stuart A. Rice, first Director of
Statistical Standards
70
Meanwhile…
• New Deal measures such as the Social Security Act of 1935
and the Fair Labor Standards Act (1937) created new
administrative mechanisms to
– Provide old age, unemployment, and disability benefits
– Set standards for minimum wages and maximum hours
• The statistical sciences went through revolutions in
measurement techniques and theory in sampling and
inferential statistics
• The decentralized system built the National Income and
Products Accounts, developed survey methods to measure
employment and unemployment, and compiled an array of
new administrative records data systems from the benefit
programs.
71
1990
1986
1982
1978
1974
1970
1966
1962
1958
1954
1950
1946
1942
1938
1934
1930
1926
1922
1918
1914
1910
1906
1902
1898
1894
1890
Unemployment, 1890-1990
35
30
25
20
Civiliian
15
Non Farm
10
5
0
ICPSR Cost of Living Survey Series
• COST OF LIVING OF INDUSTRIAL WORKERS IN
THE UNITED STATES AND EUROPE, 1888-1890
(ICPSR 7711),
• COST OF LIVING IN THE UNITED STATES, 19171919 (ICPSR 8299), and
• STUDY OF CONSUMER PURCHASES IN THE
UNITED STATES, 1935-1936 (ICPSR 8908).
73
Followed by…Consumer
Expenditure Survey Series
• The Consumer Expenditure Survey (CES) series…
provides a continuous flow of information on the
buying habits of American consumers and also
furnishes data to support periodic revisions of the
Consumer Price Index.
– (1) a quarterly Interview Survey in which each consumer
unit in the sample is interviewed every three months
over a 15-month period, and
– (2) a Diary Survey completed by the sample consumer
units for two consecutive one-week periods.
74
The Changing World of Data Analysis
• Through 1950s, only government agencies had
the capacity (technical and financial) to collect
and process replicated, geographically
complex and variable rich data sets.
• Academic or private research was limited to
secondary analysis or significantly smaller data
collections.
75
The Changing World of Data Analysis:
1960s
• The computer revolution began the process of
the ‘democratization’ of data
• Federal government began production of “public
use” microdata files
• Federal government proposed the creation of a
“data bank” for integration, and coordination of
and research using federally collected data.
• Research in “think tanks” and “contract”
organizations expanded the reach of the
statistical system
76
The World Gets Complicated Again…
• The War on Poverty and Great Society programs
expanded upon the New Deal, 1960s-1970s
–
–
–
–
–
Official poverty measure created
Medicare and Medicaid programs instituted
Aid to education
Clean Air and Clean Water Acts
Revenue sharing, 1972-86
• The reapportionment revolution and civil rights
movement restored focus on the original
formulas and ratios for apportioning political
power
77
Pressures on the Statistical System
• Small area data – for redistricting, revenue
sharing allocations
• Measuring program effectiveness and
compliance
– SIPP; civil rights compliance
• Guaranteeing census accuracy
• And since the 1980s…anti tax movements…
21st Century Challenges…
• Measuring and making policy on global
warming and climate change
• Combatting the threat of terrorism
• Monitoring the economic system and
preventing another “great recession”
Lessons from the Past?
• Science operates on its own rhythm, but
requires resources and legitimacy of the state
to bring scientific resources/knowledge to
bear on real life policy
• The political system must want a problem to
be solved, which may include being willing to
put the monitoring, analytic systems in place
to solve the problem.
Lessons…
• For both poverty measurement and
unemployment measurement, the statisticians
had a pretty good idea about what to do and how
to do it, but they had to wait for the political
system to acknowledge the issue.
• Politicians are reluctant to embed a metric in a
policy framework, particularly an automatic
trigger one, like the census apportionment
process.
• But when they do, expect the policy and the
metrics to be revisited.
Thank you. For more information…
Margo Anderson
History Department, University of
Wisconsin Milwaukee
http://www.uwm.edu/~margo
margo@uwm.edu
82