Foresight, Insight. Hindsight
US Statistical Observations
Fritz Scheuren
NORC University of
Chicago
Reminder on Definitions
• Hindsight reflecting on the
past –Personally/Collectively
• Insight, where it all comes
together, like this Conference
• Foresight future seeing or
shaping, also familiar but
bears discussion
A Small Statistical Corner
• US Official Statistics
• Censuses and Surveys
• Administrative Records
• Focus on lived experiences,
ala Deming
Times They are a Changing
• Relevance of our Discipline?
• Responsiveness of Statistics?
• Information Age?
• Misinformation Age?
• Service Partnerships?
Children’s Teaching Game?
• High?
• Low?
• You’re Too Slow!
• How Avoid Being Too Slow?
But Change is Accelerating!
• Is our Discipline Keeping Up?
• Certainly Computationally!
• Tool/Theory Building too!
• Practice Slower?
• Organizational Issues?
How To Keep Up/Catch Up?
• Google (of course)
• Metadata Revolution -- Still
more Promise than Practice
• Meta-Analytic Reuse -- Still
Often Too Hard
High-Clockspeed Trend
(use of cell phones, portable devices)
Responsiveness to Trends
• How Well Do We Play?
• High?
• Low?
• You’re Too Slow!
Response Times to Trends
(organizational clockspeed = rate at which an organization introduces new products, adopts
new production processes, or reorganizes itself;
Sources: Charles H. Fine, 1998; David W. Rejeski, 2003)
Technological Mega-Trends
• Faster and Faster Computing
(Slower for Official Statistics)
• Descriptive to Analytic
• Randomization-based to
Model-based
• Producer Dominated to
Customer Shared
Typical Grief Response to
Change Still Often True
• Denial
• Anger
• Bargaining
• Depression
• Acceptance
Examples of Hindsight,
Insight, Foresight
• Nonresponse Circa 1980
• US Census Taking Circa 1990
• Paradata Circa 2000
• Visualization Circa 2010
• Next Steps Together?
Nonresponse
Hindsight
Example
40 Year CPS Income Trend
• Insignificant Missingness in 1962
• Now Nearly Half of Interviews
have Some Missingness
• About a Third of the Amount is
Imputed
• But Still using the Same Basic
“Hot Deck” Methods Today
Greater Bias Concerns
• Possibility of Greater
Nonresponse Bias
• Potentially More Income
Understatement
• Also Characteristics of Poor
Blurred
Variance More Understated
• Growing Variance Not
Directly Reduced
• Rubin’s Multiple Imputation
Solution Still Not Used in CPS
• Remains Descriptive Rather
than Analytic
Paradata Modeling
Insight
Example
Meta-Data Revolution
• Applying Computing to
Documentation and Training
• Including Measurement
Process or Paradata
• Achieving Full Systems
Thinking
Unify Meta/Paradata
• Bringing All Survey Aspects
together electronically
• Sharing with All Stakeholders
• Breaking Down Barriers
between Departments
Unify Meta/Paradata
• Bringing All Survey MetaData together electronically
• Sharing with All Stakeholders
• Breaking Down All Barriers
Between Departments
Manage System as a Whole
• Not Just Conformance to
Requirements Quality
• But Total Fitness for Use
Quality
• From Sampling/Nonsampling
to Total Survey Inference
• Record linkage Example
Total Systems Thinking
• Turning Sample “Models”
• Into Full Survey “Models”
• Using Paradata and
Experience
• Politz-Simmons Example
Visualization
Foresight
Example
Complex Survey Graphics
• Clustering and Weighting
Distort
• Analytically these can be
“solved” Approximately
• Design Effect Example
Restoring Visual Metaphor
• Inverse Sampling Algorithm
• Works for Many Designs
• Satisfactory Analytically
• Works Graphically too but
not yet Always
Simulation Alternatives
• Capture Essential
(Sufficient?) Conditions
• Simulate Graphics
Analytically
• Retain Real Sample?
• Apply Empirical Residuals?
Regression Diagnostics
• Design-Based and Analytic
Alternatives Being Examined
Now
• Both Have Merits
• May Imbed both in an OpenSource, like R
Next Steps?
Further Considerations
and Examples
Expected White Swan?
Unexpected Black
Swan
Addressing Evolving Trends