Public Health Systems Research
Interest Group Breakfast
Interpreting Public Health Rankings
Sponsored by the Robert Wood Johnson Foundation
http://www.academyhealth.org/phsr
February 2010
Ready or Not?
Using Data for Public
Health Advocacy
Jeffrey Levi, PhD
Trust for America’s Health
February 9, 2010
Academy Health Conference
Who We Are
†
Trust for America’s Health (TFAH) is a
non-profit, non-partisan organization
dedicated to saving lives by protecting
the health of every community and
working to make disease prevention a
national priority.
Preparing for Public Health Emergencies
Goals of the Report
†
Snapshot ranking of the states: the frontlines
of preparedness
„
„
„
Accountability for billions of dollars in federal
funding
Show where some of the gaps are
Push states to adopt improved policies
Why rank?
†
†
It gets everyone’s attention
States seek to improve their scores
„
„
†
Some state preparedness officials use scores to
prod their legislatures and Governors
Other states see scores as unfair punishment
Congressional staff want to know how their
states are doing – and what can be done to
help
„
Often translates into interest in increased funding
Biggest challenge: State-by-state data
†
†
†
†
Federal government has been slow to set
performance indicators
States do not report information in a similar way
Not all states want to cooperate in ranking ventures
Left to use mostly publicly available data
„
Has it limits – but alternative is no accountability at all
2009 Report State Indicators
†
†
†
†
†
Mass Distribution –
Antivirals
Hospital Bed
Availability Reporting
Public Health Labs –
Pickup and Delivery
Public Health Labs –
Surge Workforce
Biosurveillance
†
†
†
†
†
Food Safety –
Detection & Diagnosis
Medical Reserve Corps
Readiness
Children &
Preparedness
Entity Liability
Protection
Funding Commitment
Scores by State
Toughest criticism
†
†
Variability in scores over time
We do change the indicators –
„
„
†
We retire indicators where overwhelming achievement
(but continue to show the progress)
We add indicators based on new areas of interest
Are the indicators true reflections of preparedness?
„
„
Use of surrogates
Even some that are “true” indicators are not necessarily
good predictors (SNS scores vs. H1N1 vaccine
distribution)
Mass Distribution – Strategic National
Stockpile
Does the state have an adequate SNS plan?
60
50
51
51
2008
2009
41
40
30
20
10
0
15
7
2005
2006
2007
State Antiviral Purchases
Did the state purchase ≥ 50% of its federallysubsidized antivirals?
40
35
30
25
20
15
10
5
0
35
38
28
2007
2008
2009
Biosurveillance -- NEDSS
Does the state use a disease surveillance system that is
NEDSS-compliant?
50
40
30
20
38
39
2006
2007
45
45
2008
2009
27
18
10
0
2004
2005
Legal Preparedness – Health Care
Volunteer Liability Protection
Does the state have liability protections
for emergency health care volunteers?
50
43
43
2008
2009
40
30
30
20
10
0
2007
Is the juice worth the squeeze?
†
†
†
Do rankings of this nature highlight issues
enough to promote policy change?
Do rankings of this nature create bad feelings
that reduce cooperation on other issues?
What would it take to get us out of this
business?
„
Real performance measures released by the
federal government
The County Health Rankings
Mobilizing Action Toward Community Health
Brenda Liz Henry, PhD, MPH
AcademyHealth
National Health Policy Conference
February 9,
9, 2010
County Health Rankings:
A Call to Action to Improve Health
Who?
The Robert Wood Johnson Foundation (RWJF) and The University of Wisconsin
Population Health Institute (UWPHI).
What?
First-of-its-kind collection of 50 reports – one per state – that ranks all counties
within each state on their overall health.
When?
February 17, 2010.
Where?
www.countyhealthrankings.org
30
AcademyHealth – National Health Policy Conference
February 9, 2010
Precursor to the County Health Rankings
Building on America’s Health Rankings which ranks the health of
the 50 states, the University of Wisconsin began ranking the
health of Wisconsin’s counties in 2003.
31
AcademyHealth – National Health Policy Conference
February 9, 2010
What is the model behind the County Health Rankings?
32
AcademyHealth – National Health Policy Conference
February 9, 2010
How were measures selected for the County Health
Rankings?
• Reflect important aspects of population health that can be improved
• Valid, reliable, recognized and used by others
• Available at the county-level
• As up-to-date as possible
• Fewer measures better than more
33
AcademyHealth – National Health Policy Conference
February 9, 2010
Health Outcomes: Mortality and Morbidity
Measuring Length and Quality of Life
Construct
Indicator
Mortality (50%)
Premature death – years of life lost
before age 75
Sources: Vital Statistics, National Center for Health
Statistics (NCHS)
Morbidity (50%)
Self-reported poor or fair health
Physically unhealthy days
Mentally unhealthy days
Low birth weight
Sources: Behavioral Risk Factor Surveillance
System (BRFSS), Vital Statistics, NCHS
34
AcademyHealth – National Health Policy Conference
February 9, 2010
Health Determinants: Health Behaviors (30%)
Construct
Indicator
Tobacco Use
Smoking Rate
Source: BRFSS
Diet & Exercise
Obesity Rate
Source: BRFSS
Alcohol Use
Binge Drinking Rate
Source: BRFSS
Sexual Behavior
Sexually Transmitted Disease Rate,
Teen Birth Rate
Sources: Center for Disease Control and
Prevention (CDC), National Center for Hepatitis,
HIV, STD, and TB Prevention, Vital Statistics,
NCHS
Driving
Deaths Due to Motor Vehicle Crashes
Sources: Vital Statistics, NCHS
35
AcademyHealth – National Health Policy Conference
February 9, 2010
Health Determinants: Clinical Care (20%)
Construct
Indicator
Access to Care
People Without Insurance, Primary
Care Providers
Sources: Census/Current Population Survey (CPS)
– Small Area Health Insurance Estimates (SAHIE),
Health Resources and Services Administration,
Area Resource File (ARF)
Quality of Care
Hospital Stays for Ambulatory Care
Sensitive Conditions, Diabetics that
Receive HbA1C Screening, Hospice
Care in Last 6 Months of Life
Sources: Medicare Claims/Dartmouth Atlas
36
AcademyHealth – National Health Policy Conference
February 9, 2010
Health Determinants: Social and Economic Factors (40%)
Construct
Indicator
Education
High School Graduation Rate, Adults
with College Degree
Sources: National Center for Education Statistics,
Decennial Census, American Community Survey
(ACS)
Employment
Unemployment Rate
Source: Local Area Unemployment Statistics
Income
Children in Poverty, Income Inequality
Sources: Census/CPS, Small Area Income and
Poverty Estimates (SAIPE), Bureau of Labor
Statistics, Decennial Census, ACS
Family & Social Support
Social/Emotional Support, Single-Parent
Households
Sources: BRFSS, Decennial Census, ACS
Community Safety
Violent Crime Rate
Sources: Uniform Crime Reporting, FBI
37
AcademyHealth – National Health Policy Conference
February 9, 2010
Health Determinants: Physical Environment (10%)
Construct
Indicator
Air Quality
Unhealthy Air Quality Days Due to
Ozone, Unhealthy Air Quality Days Due
to Fine Particulate Matter
Source: CDC-Environmental Protection Agency
(EPA) Collaboration
Built Environment
Zip Codes Without Healthy Food
Outlets, Liquor Store Density, Housing
Stock Built Before 1950 (proxy for
exposure to lead)
Sources: Census Zip Code Business Patterns,
Census County Business Patterns and Census
2006 Population Estimates, Decennial Census,
ACS
38
AcademyHealth – National Health Policy Conference
February 9, 2010
What’s Unique about the County Health Rankings?
• This is the first time we will have a standard measure of the overall health
of each county in every state across the country.
• Every state can see how its counties compare on multiple factors that
influence health.
• Every county will receive a snapshot of how healthy its residents are and
how factors such as behavior, education and environment influence health
in that commmunity.
• The County Health Rankings can mobilize community leaders to see what
is making their residents sick or unhealthy and develop solutions to give
everyone healthier choices.
39
AcademyHealth – National Health Policy Conference
February 9, 2010
Why County Health Rankings?
Where we live matters to our health.
• There are big differences in health across communities, and some places are
much healthier than others.
Some communities are less healthy than others because they have
not removed obstacles that we know keep residents from making
healthier choices.
• Having a report that ranks our county’s health helps us identify our problems and
helps us find solutions so all residents can be healthy.
Health is everyone’s responsibility.
• Leaders from every sector must recognize what factors beyond medical care
influence the health of their communities so they can work together to create
programs and policies to help people lead healthier lives.
40
AcademyHealth – National Health Policy Conference
February 9, 2010
through
County Health Rankings
Setting goals
and objectives
for overall
health
Keeping track
of progress
MATCH model © 2010 UWPHI
41
AcademyHealth – National Health Policy Conference
February 9, 2010
County Health Rankings in Wisconsin:
A Call to Action in Juneau
In 2006, the University of Wisconsin ranked Juneau as the unhealthiest
county in Wisconsin – 73rd out of 73 counties.
42
AcademyHealth – National Health Policy Conference
February 9, 2010
County Health Rankings in Wisconsin:
A Call to Action in Juneau
After the Wisconsin Rankings came out, the community in Juneau
County mobilized. Today, Juneau County is implementing plans to
address parenting, health literacy, and health care access – all factors
that contributed to their low health ranking.
43
AcademyHealth – National Health Policy Conference
February 9, 2010
Using the County Health Rankings
State and Local Policymakers
• Can use the Rankings to push for policy changes to improve health
State and Local Public Health Departments
• Can use the Rankings to find out what is making their residents sick, and
to secure funding for programs to address those local public health
problems
Community Members
• Can share the Rankings with stakeholders who are positioned to
mobilize local leaders to make real change
44
AcademyHealth – National Health Policy Conference
February 9, 2010
County Health Rankings:
Coming Soon to a County Near You!
County Health Rankings will be released in each
of the 50 states on February 17, 2010
For more information:
www.countyhealthrankings.org
45
AcademyHealth – National Health Policy Conference
February 9, 2010
State and local health rankings
Michael A. Stoto
Georgetown University
February 9, 2010
Measurement preliminaries
• Measurement development cycle
– Clarify purpose
– Identify concepts to be measured
– Develop specific indicators
– Assess validity and reliability
• From the “social indicators” movement
– a limited yet comprehensive set
– of coherent and significant indicators
– that can be monitored over time and
– disaggregated to relevant social units
Why prepare state and local
health indicators?
• Accountability
– Federal preparedness funding
– State and local health departments
– Other parts of the public health system
• Quality improvement
– Publicly identify low performers to prod action
– Guide internal quality improvement efforts
• May require different indicators
IOM CHIP model
Improving Health in
the Community, 1997
What should be measured?
• Health profile
– Comprehensive measures: Premature mortality
– Specific measures: Low birthweight
– Health risks: Smoking, obesity
– Social environment: Unemployment rate
• Performance measures
– Healthcare access and quality indicators
– Public health and other specific entities
– Response capabilities, not just preparedness
capacities
Validity
• Do indicators measure the right concepts?
– Determinants of health
• Are response capabilities associated with
preparedness capacities?
– Important dimensions of health outcomes
• Weights – importance and contribution
– Locus of responsibility
• Shared responsibilities
• County as unit
• Regional issues
– “Is my issue/program included?”
– Teaching to the test
Reliability
• Do indicators measure concepts
consistently?
– Is measurement error small compared to
population variability?
– Will the indicator change if and only if the
concept being measured changes?
• Inherent variability in small area statistics
• Changes in definitions and measurement
process
Example: TFAH rankings
Is this due to
TFAH State Scores 2007 & 2008
10
2008 sco re
9
8
7
6
5
5
6
7
8
9
2007 score
r = 0.235
10
• Improvements and
degradations in
preparedness?
• Differences in
indicators included in
the overall score
• Variability in the
measurement
process
Why rank?
• Not “why produce state/county level health
data?”
• Rankings too variable
– especially for small areas
– may not draw attention to all of the
jurisdictions that need it
• Every jurisdiction has strengths and
weaknesses
• Is this the best approach for quality
improvement?
• Are there alternative ways to benchmark?