Effective MCH
Epidemiology
Translation and
Use?
HRSA/CDC MCH Epidemiology Course
Dr. William Sappenfield
September 11, 2012 Webinar
Classic Definition of Epidemiology
“Epidemiology is the study of the distribution and
determinants of health-related states or events in
specified populations, and the application of this
study to the control of health problems.”
J. M. Last
Definition of MCH Epidemiology
“The systematic collection, analysis and
interpretation of population-based and programspecific health and related data in order to assess
the distribution and determinants of the health
status and needs of the maternal child population
for the purpose of planning, implementing, and
assessing effective, science-based strategies and
promoting policy development.”
Coalition for Excellence in MCH Epidemiology, 2010
Definition of MCH Epidemiology
“The systematic collection, analysis and
interpretation of population-based and programspecific health and related data in order to assess
the distribution and determinants of the health
status and needs of the maternal child population
for the purpose of planning, implementing, and
assessing effective, science-based strategies and
promoting policy development.”
Coalition for Excellence in MCH Epidemiology, 2010
Effective MCH EPI Practice
Results Focused
Data Use “Triangle”
Data & Analysis
TRANSLATION
Planning &
Programs
Community
Politics &
Policy
According to a past survey of
evaluators, what is the major problem
related to program evaluation?
According to a past survey of
evaluators, what is the major problem
related to program evaluation?
Using the results!
Common Reasons for Not Using
Evaluation Results
Stakeholders are not engaged
 Answering the wrong questions
 Evaluator losing creditability
 Not reading/understanding the results
 Not liking the results
 Not wanting change

Epidemiology Training
Functions
 Distribution
 Determinants
Focus
 New knowledge
 Confirmation
Descriptive
Analytic
Seeing the Ends
Consequential Epidemiology
Policies & Programs
Intervention Epidemiology
Policies & Programs
Being Effective in Public Health
Data Connections to Planning Cycle
Awareness
5 Common Mistakes
“For we break, we’re
going to let the statistics
speak for themselves.”
Being Effective in Public Health
Time Required to Use Results
"We must become the change
we wish to see in the world."
Mahatma Gandhi
Effective MCH EPI Practice
Conceptual Framework
MCH Epidemiology Efforts
M
A
C
R
O
C
O
N
T
E
X
T
Health
Agency
Structure
Population
Context
Public
Health
Context
The capacity to do work:
the work environment
MCH-Epi
Effort
Structure
MCH-Epi
Effort
Process
Many processes and
outputs become
structural features
Practice Activities:
The work that is done
Health
Status
Practice Results:
The results of that work
Output
Intermediate
Outcome
Conceptual Framework
MCH Epidemiology Efforts
M
A
C
R
O
C
O
N
T
E
X
T
Health
Agency
Structure
Population
Context
Public
Health
Context
1. Includes sufficient funding for continuing
education and staff development
2. Includes database and web servers,
statistical analysis software, GIS
software, etc.
1. Includes a lead MCH epidemiologist
2. Includes sufficient staff for data
collection, analysis, dissemination, etc.
3. Participates in the leadership of the
agency
MCH-Epi 1. Provides analytic direction
Health
2. Provides expertise for data
Effort
Status
system development
Process 3. Conducts high-level data
Feedback Loop
analysis
1.
Data processes
linkage /
Many
and
integration
outputs
become
2. Grant submission and
structural features
Intermediate
subsequent funding
Output
Outcome
1. Disseminates
1. Program and policy
reports
change, with or
2. Generates policy
without legislation
briefs
MCH-Epi
Effort
Structure
Multivariable Models Across Domains
Final Recommendations to States
Establish a named unit for MCH
Epidemiology.
 Ensure its leadership has organizational
recognition & authority.
 Acknowledge its broad scope and
collaborative approach.
 Hire increasing numbers with doctoral
degrees.
 Invest in a critical mass of MCH
epidemiologists.
 Pursue assignees, fellows, & interns.

Final Recommendations to States (2)
Provide staff with time and funding for
training.
 Ensure direct access to a wide variety of
datasets.
 Routinely link data beyond birth-death data
 Disseminate the work using multiple
approaches/venues.
 Jointly translate findings into information for
action.
 Support external partners turning data into
information.

Consequential Epidemiology
Policies & Programs
Deaths Per 1,000 Live Births .
Infant Mortality Rates
Florida and U.S., 1975 to 2005
20.0
15.0
10.0
5.0
0.0
FL
US
10.0
8.0
6.0
4.0
FL
US
2.0
20
05
20
04
20
03
20
02
20
01
20
00
19
99
19
98
19
97
19
96
0.0
19
95
Deaths Per 1,000 Live Births .
Infant Mortality Rates
Florida and U.S., 1995 to 2005
Support during pregnancy for women at
increased risk of low birthweight babies
“Pregnant women need the support of
caring family members, friends, and health
professionals. While programs which offer
additional support during pregnancy are
unlikely to prevent the pregnancy from
resulting in a low birthweight or preterm
baby, they may be helpful in reducing the
likelihood of caesarean birth.”
Hodnett, Cochrane 2003
Draft Report Findings:
Economic & Outcome Evaluation
of the Florida Healthy Start
Stephanie Staras, Ph.D.
John Kairalla, Ph.D.
Elizabeth Shenkman, Ph.D.
Institute for Child Health Policy
Department of Epidemiology and Healthy Policy Research
College of Medicine, University of Florida
Summarized by William M. Sappenfield, MD, MPH
Study Questions


How do Healthy Start prenatal clients in high
benchmark coalitions compare to Healthy Start
clients low benchmark coalitions?
o Preterm, post-term, LBW, and SGA
o Timely prenatal care and receiving postpartum
care
o Medicaid expenditures
How do Healthy Start prenatal clients compare
to non-clients in high and low benchmark
coalitions?
o Same outcomes and expenditures
36
Methods



Data: Merged files for 1998-2006 including
o Live birth certificates
o Healthy Start prenatal screens
o Healthy Start prenatal services
o Medicaid eligibility and claims
Design: Retrospective Observational Cohort
Study Population: All Florida resident women
with a live birth
37
Methods—Services
Women receiving Healthy Start prenatal
services were collapsed into the following
service categories:
 Initial Contact
 Initial Assessment
 Care Coordination
 Supplemental Services
38
Methods—Healthy Start Study
Categories
All Study Women: Women
 With singleton live births
 Covered by Medicaid (not medically needy)
 Healthy Start screen of four or more
Healthy Start Care Coordination: Women who
 Received a care coordination service
No Healthy Start Service: Women who
 Received no Healthy Start services
39
Methods—Healthy Start High and
Low Benchmark Coalitions
Eight Highest & Lowest Benchmark Coalitions:
 Prenatal screening rate
 Percent of women served prenatally
 Mean number of Healthy Start services
Of eight selection methods explored
 Same 8 low benchmark coalitions selected
 Same 6 high benchmark coalitions selected
40
Methods—Comparisons
HS in High Benchmark Coalitions
First
HS in Low Benchmark Coalitions
High Benchmark Coalitions—HS
Second
High Benchmark Coalitions—No HS
Low Benchmark Coalitions—HS
Third
Low Benchmark Coalitions—No HS
41
Methods—Time Periods
Time Period
Start
End
Pre-waiver
Jan 1, 1998
Transition period
July 2, 2001 March 23, 2002
Early post waiver March 24, 2002
Late post waiver
July 2, 2004
July 1, 2001
July 1, 2004
Dec 31, 2006
42
Methods—Statistical Methods


Observational Retrospective Cohort Analysis
SAS
o Expenditures—Mixed model
o Birth Outcomes and Care—Generalized
estimating equations (GEE)
o Manage repetitive events
o Provide multilevel adjustment
43
Differences in the Percentages or Expenditures
for Three Healthy Start Comparisons
1
If Healthy Start were effective…
Difference in Percents
0.5
0
-0.5
-1
Ideal
-1.5
-2
-2.5
-3
Pre-waiver
Transition
Early Post
Late Post
Differences in the Percent Preterm Births
for Three Healthy Start Comparisons
3
Difference in Percents
2
1
0
HS--High to Low
High--HS to No
Low--HS to No
-1
-2
-3
-4
-5
Pre-waiver
Transition
Early Post
Late Post
Differences in the Percent Low Birthweight
Births for Three Healthy Start Comparisons
3.5
Difference in Percents
3
2.5
2
HS--High to Low
High--HS to No
Low--HS to No
1.5
1
0.5
0
-0.5
-1
Pre-waiver Transition
Early Post
Late Post
Differences in the Percent Small for Gestational
Age Births for Three Healthy Start Comparisons
5
Difference in Percents
4
3
2
HS--High to Low
High--HS to No
Low--HS to No
1
0
-1
-2
-3
-4
Pre-waiver
Transition
Early Post
Late Post
Differences in the Percent Receiving Timely
Prenatal Care for 3 Healthy Start Comparisons
Difference in Percents
10
8
6
4
2
HS--High to Low
High--HS to No
Low--HS to No
0
-2
-4
-6
-8
-10
Pre-waiver
Transition
Early Post
Late Post
Differences in the Percent Receiving Postpartum
Care for 3 Healthy Start Comparisons
2
Difference in Percents
1
0
HS--High to Low
High--HS to No
Low--HS to No
-1
-2
-3
-4
-5
Pre-waiver
Transition
Early Post
Late Post
Differences in Adjusted Expenditures for the
Prenatal Period for 3 Healthy Start Comparisons
Difference in Dollar Expenditures
1200
1000
800
HS--High to Low
High--HS to No
Low--HS to No
600
400
200
0
Pre-waiver Transition
Early Post
Late Post
Differences in Adjusted Expenditures for the
Delivery for 3 Healthy Start Comparisons
Difference in Dollar Expenditures
100
0
-100
HS--High to Low
High--HS to No
Low--HS to No
-200
-300
-400
-500
-600
Pre-waiver Transition
Early Post
Late Post
Differences in Adjusted Expenditures for Infants
(1-12 months) for 3 Healthy Start Comparisons
Difference in Dollar Expenditures
1200
1000
800
HS--High to Low
High--HS to No
Low--HS to No
600
400
200
0
-200
Pre-waiver Transition
Early Post
Late Post
Evaluation Limitations
Observational study—no randomization of
Healthy Start prenatal services
o Unresolved selection bias / confounding
 Secondary analysis—not designed directly
 Statistical power—low for some outcomes
 Accuracy of linkage—unlinked or incorrect
 Accuracy reporting—vital records
 Change in reporting—birth certificates

53
Evaluation Strengths






Examines outcomes and medical expenditures
Separates care coordination and prenatal score of
4+ from other Healthy Start services
Comparable independently-ascertained outcome
measures
Uses multiple comparisons including high and low
benchmark coalitions
Adjusts for additional medical confounders using
Medicaid claims
Adjusts for county level differences
54
Conclusions—Services

Care Provision: Some low risk women are
receiving care coordination while some high risk
women are not

Coalitions: High variability between coalition
sites in the provision of care coordination by risk
level

Variability: Variability in program
implementation may have contributed to the
inability to detect differences
55
Conclusions—Outcomes

Health Outcomes: Healthy Start did not show a
consistent pattern of improved preterm, low
birthweight and small-for-gestational-age births

Services: Healthy Start did not show a
consistent pattern of improved timely prenatal
care and postpartum services

Expenditures: Healthy Start demonstrated
consistent reduced expenditures for delivery
care, but less than the increased expenditures in
the prenatal period and infancy (1-12 months)
56
Conclusions

Healthy Start does not appear to be impacting
birth outcomes and health care costs. This is
consistent with most of the literature regarding
pregnancy support services.
57
Questions?
Evaluation of Healthy
Start Prenatal Services:
A Focus on Immediate Outcomes
William Sappenfield, MD, MPH
Leticia Hernandez, PhD, MS
Dan Thompson, MPH
Cheryl Clark, DrPH, RHIA
Division of Family Health Services
Past Healthy Start
Evaluations
Healthy
Start
Screen
LBW &
Infant
Mortality
Healthy
Start
Services
Medicaid
Healthy Start
Mother’s EligibilityUF
Prenatal ScreenUF
Prenatal ServicesUF
Live Birth
Certificate
WIC
Prenatal
Past Healthy Start
Evaluations
Healthy
Start
Screen
Healthy
Start
Services
Health Ed.
Quit Smoking
Sleep Position
Breast Feeding
Contraception
LBW &
Infant
Mortality
Medicaid
Healthy Start
Mother’s EligibilityUF
Prenatal ScreenUF
Prenatal ServicesUF
Live Birth
Certificate
WIC
PRAMS
Prenatal
Maternal Survey
Study Question
What is the association of Florida Healthy
Start (HS) prenatal services and the maternal
and infant health behaviors and experiences?
Prenatal / Perinatal
Post Partum
 Adequate prenatal visits
 Postpartum contraception
 Prenatal counseling
 Health provider in first week
 Prenatal WIC participation  Breastfeeding
 Gestational weight gain
 Sleep position & location
 NICU admissions
 Passive smoking
62
Methods



Data: Merged files for 2000-2005 including
o PRAMS
o Live birth certificates
o Healthy Start prenatal screens
o Healthy Start prenatal services
o WIC Program files
Design: Retrospective Cohort
Study Population: All women that answered
the PRAMS survey
63
Methods—Services
Women receiving Healthy Start prenatal
services were collapsed into the following
service categories:
 Initial Contact
 Initial Assessment
 Care Coordination
 Supplemental Services
64
Methods—Healthy Start
Intervention Groups
HS Care Coordination: Women who
 Said ‘yes’ to the screen consent and
 Received an initial assessment and/or a
care coordination service
Other HS Services: Women who
 Said ‘yes’ to the screen consent and
 Received only a HS initial contact and/or
HS supplemental services
65
Methods—Healthy Start
Comparison Groups
Women screened no services: Women who
 said ‘yes’ to the screen consent, and
 did not receive services
Women not screened: Women who
 said ‘no’ or not offered the screen, and
 did not receive services
66
Methods—Models
HS Care Coordination
Model 1
Women Screened No Services
HS Care Coordination
Model 2
Women Not Screened
67
Methods—Statistical Methods
Weighted to provide state-level estimates
 Binomial regression
o STATA SE, V.9.2

68
Summary—Evaluation Findings
Prenatal / Perinatal
Post Partum
Positive
Positive
• Adequate prenatal visits
• Breastfeeding initiation
• Breastfeeding duration
• Prenatal counseling
• Prenatal WIC participation
Neutral
Neutral
• Postpartum contraception
• No inadequate weight gain • Health provider in first week
• No excessive weight gain • Back sleep
• No NICU admissions
• No bed sharing
69
• No passive smoking
Evaluation Limitations
Observational study—no randomization of
Healthy Start prenatal services
 Secondary analysis—not designed directly
 Program integrity—32 coalitions over 6 yrs
 Statistical power—low for some outcomes
 Accuracy of linkage—unlinked & incorrect
 Accuracy reporting—vital records & PRAMS
 Change in reporting—birth certificates

70
Evaluation Strengths
Examines direct short-term program effects
 Separates care coordination from other
Healthy Start services
 Comparable independently-ascertained
outcome measures
 Two different reference groups for exploring
potential selection bias
 Adjusts for additional confounders using
PRAMS

71
Conclusions

Healthy Start improves some pregnant
women and infant health outcomes—small
level of effect.

Healthy Start may benefit from program
enhancements and increasing intensity.

Further evaluation studies on health
service utilization and impact.
72
Questions?