State Health Surveys in the U.S.:
Their Characteristics and
Innovative Leadership
Jennifer Kincheloe, MPH, PhD
on behalf of collaborating team from:
UCLA Center for Health Policy Research
National Cancer Institute
National Heart, Lung, and Blood Institute
National Institute on Aging
NIH Office of Disease Prevention
AcademyHealth
State Health Research and Policy Interest Group
June 26, 2010
www.healthpolicy.ucla.edu
Study Purpose and Design
X NCI and other NIH agencies sponsored study to
understand more about state and local surveys
f Identify health surveys that can help meet
research needs for local data
X Study included two sets of tasks
f Task I: Develop inventory of state-based population health
surveys and identify the comprehensiveness of topics covered,
whether they provide local level data, and other characteristics
f Task II: Interview leaders of sample of independent state and
local health surveys to identify core elements of successful
surveys
Task I Methods – Systematic Inventory
X Examined all 50 states
f Recent and periodic surveys
f Comprehensive scope: demographics, health status, chronic
conditions, health behaviors, risk factors, health insurance
X Reviewed public Web sites for:
f Survey descriptions, instruments, technical reports, policy
publications
f Limited follow-up by e-mail or phone
f Collected topics, subpopulations, technical aspects,
dissemination strategies, innovations
X Findings reflect most recent information available at
the time of data collection
Study Task II - Methods
X Eight states participated (CA, AZ, IL, NC, IA, OH, UT,
MO), plus NYC
f Geographically, ethnically representative
f Representative of innovations
X Telephone interviews in participating states
f 16 high-level survey staff in universities, foundations, hospitals,
and state health departments
f Recorded and transcribed interviews
BRFSS:
Behavioral Risk Factor Surveillance System
X All states and US territories have the BRFSS
f Annual survey of adults
f Basic sample designed to produce
state-level estimates
X “SMART” estimates produced by CDC when
MMSA/county sample size is sufficient
f MMSA sample> 499 and samples of subgroups used in weighting
are adequate
f Estimates are not based on sample drawn to represent the
MSA/county
BRFSS:
Variations across states
X All states add optional modules to their BRFSS
f 8 states add their own questions
X 16 states add local strata to generate their own,
representative local estimates
X 3 states field additional child/adolescent surveys
linked to BRFSS parents (CO, NC, IL)
Independent Health Surveys
X Some states established independent surveys
because BRFSS did not adequately meet local and
state data needs
f California Health Interview Survey released
first data in 2002
f Extensively used for policy development
and advocacy
f Researchers make extensive use of data
f Sustaining support from multiple public and private funders
X Some other states initiated similar statewide local
health surveys
Independent Health Surveys
X Independent health surveys are conducted in 11
states
f CA, AZ, OH – Separate adult survey, plus adolescent and/or
child surveys
f IO – Separate child survey
f HI, UT – One interview on behalf of the household
f MO – One randomly selected adult
f NJ – One adult interviewed about family members, including
about a dependent young adult age 19-30 not living at home
f IL, CO, NC – Adolescent and/or child surveys conducted with
sample drawn from BRFSS adults
X Most states adapt questions from federal surveys
X Sample sizes: >50,000 households to 2,500
BRFSS and Independent State Health Surveys
BRFSS & Independent Surveys:
Wide Technical Variation
X Local samples and estimates
f BRFSS: Only some states stratify sampling at local level
f BRFSS: Some states produce local estimates from samples that
may not be based on post-survey stratified weights
f SMART BRFSS: CDC produces estimates based on post-survey
stratified weights
f Independent surveys: Local estimates based on local strata
(except NC) ranging from neighborhood to regional levels
BRFSS & Independent Surveys:
Wide Technical Variation
X Source of survey questions
f BRFSS: Questions provided by CDC and some new questions
are developed by state
f Independent surveys: Questions drawn from established surveys
and some new questions are developed
X Response rates
f BRFSS: Standardized response rates coordinated by CDC
f Independent surveys: States currently set own rules about how
response rates are calculated
BRFSS & Independent Surveys:
Wide Technical Variation
X Statistical precision and stability of estimates
f BRFSS: No uniform standard by states to decide whether
estimates are stable
f BRFSS: CDC SMART website uses standard criteria for
evaluating estimates
f Independent surveys: Currently no uniform standard by states to
decide whether estimates are stable
X Innovations
f Independent surveys: Experiment with innovations that could be
shared/replicated for other state health surveys
f BRFSS: CDC experiments and introduces some innovations
Dissemination of State Health Survey Data
X Wide variation in extent of dissemination
f A few make downloadable data files available
f Some have dynamic survey query systems
f Most provide survey estimates from pre-run tables
f Some conduct outreach/trainings to potential data users
X California Health Interview Survey
f Most extensive dissemination – invests one in five project dollars in
dissemination
f AskCHIS, user-friendly query system
State and Local Health Survey Data Widely Used
X Data are used by diverse stakeholders, indicating a
broad demand for data:
f Researchers in universities, governments, and private
organizations
f State and local public health agencies
f State and local policy makers, including governors, legislatures,
city council, and county supervisors
f Community-based organizations and advocacy groups
f Health coalitions and taskforces, including health care reform
task forces
f Funders
f News media
State and Local Health Survey Data Widely Used
X Survey data are used for multiple purposes:
f
f
f
f
f
f
f
f
f
f
f
Policy or research briefs and reports
Health profiles of neighborhoods, counties and states
Health advocacy
Policy decisions
Analysis of health trends
Surveillance
Program development and evaluation
To support fundraising
Academic research
News media
Conferences and presentations
Task II: Starting an Independent Health Survey
X Initiated because BRFSS was not meeting need for:
f Local data based on representative samples
f Data on children and adolescents, preferably linked with data
on parents
f In-depth data on topics of interest to states
X Initiated by variety of public and/or private entities:
f Universities
f State and local health departments
f Foundations
f Hospitals and clinics
Task II: Starting an Independent Health Survey
X Varied planning
f The extent of planning and needs assessment determined by
resources and any timelines imposed by funders.
X Strong leadership
f Each survey had at least one champion to push for its creation
Task II: Funding of Independent Health Surveys
X Funding came from multiple sources, indicating
broad support for independent health surveys
f Foundations
f County, city, state and federal government agencies
f State legislatures
f Federal block grants and federal contracts and grants
f Tobacco settlement monies
f Hospitals
X Funders play a key role in determining state health
survey content, characteristics, and dissemination
Task II: Independent Survey Leaders Seek
X Network of independent health surveys to share
resources, information, experiences, and innovations
f Effort to harmonize question sets across surveys, without
sacrificing state priorities
f Minimum quality standards and best practices
f Technical support to deal with emerging challenges in survey
research
X Consistent funding to help sustain their efforts
Conclusions
X Great variation among independent health surveys
f Limited comparability among health measures
f BRFSS uses more comparable measures
f More comparability would enhance research utility of data
X Growing interest in local data provides opportunity to
foster quality, innovation and comparability by:
f Facilitating sharing of information and experiences
f Providing funding, technical assistance, and other resources
f Engaging states in harmonization of measures and survey
approaches
f Developing best practices and/or minimum quality standards