Evaluating Physical Activity
Intervention Programs
Thomas Schmid, PhD
Physical Activity and Health Branch
CDC
Atlanta, Georgia, USA



Evaluation Definitions and
Principles
CDC Physical Activity
Evaluation Handbook – 6
Steps
Evaluation issues
The systematic examination and
assessment of features of a
program and its effects. Evaluation
is designed to produce information
that can be used by those who have
an interest in the improvement or
effectiveness of an initiative or
program.
 To
assess the effects and value of a
program
◦ for accountability
◦ for progress
◦ to identify best practice
◦ as a planning tool
◦ for research




the program
target groups and needs
realistic achievable objectives
is the program ready for evaluation?

Identify needs

State goals and measurable objectives

Assess progress towards objectives
Efficacy
Intervention tested under ideal conditions
with optimal study design
Effectivenes
Field based intervention testing whether the
intervention works as intended for a defined
population in the real world
Efficiency
Results in comparison to the efforts required
to obtain them – often cost or cost
effectiveness
Formative
Measures short term
objectives
IMPACT
Develop program
& resources
OUTCOME
PROCESS
Measures activities,
program quality, reach
Measures long
term effects
(goals)





What is likely to work
Best practices in health promotion
Idea or concept development
Developing and pilot testing materials
Focus groups



Focus groups and
interviews with
partners about their
needs
Reviewing best
practices from
other programs
such as Agita Sao
Paulo
Pilot testing in one
company
Process evaluation
◦ Program implemented as intended ?
◦ Quality of program components
◦ Attendance rates
◦ Audience awareness of message
◦ Appropriate message learned
◦ Participant satisfaction

Companies enrolled

Workshops held

Promoters trained

Companies
participating in
community events
Impact & Outcome
Evaluation
◦
Short and long term effects of program
◦ Hierarchy of effects : what level of effects indicates
program success
◦ Assessing the impact upon those who attend or
participate
◦ Community effects

Physical activity level
of employees

PA level in community

Disease rates or costs
Program Evaluation Framework
Physical Activity Evaluation Handbook
Program Evaluation Framework
Physical Activity Evaluation Handbook
1. Engage Stakeholders
Include people
with a vested interest in the program:
1. Engage
Stakeholders
implementers, partners, participants, possible
opponents and decision makers.
2. Clearly Describe the Program
The problem being addressed, descriptions of the
planning, implementation and maintenance phases of
the program, and finally, defining and communicating
how all phases work together (eg. the "logic model").
Program Evaluation Framework
Physical Activity Evaluation Handbook
3. Clearly Focus
3. Clearly
AskingFocus
a clear, straightforward question and
ensuring linkage to the problem being
addressed. Includes clarifying process and/or
outcome evaluations of interest
4. Develop and Implement Data Collection
Design of data collection efforts, selection of
appropriate control groups, designation of
appropriate baseline values and decisions about
how much time should be built in to adequately
measure "change”
Program Evaluation Framework
Physical Activity Evaluation Handbook
5. Develop Conclusions
5. Develop Conclusions
Data analyses and interpretation.
6. Communicate
Promotes the importance of tangible products from the
evaluation, dissemination of those products, and follow-up (as
appropriate).
www.iuhpe.org
20




All physical activity interventions should be
evaluated. Evaluation begins with program
planning
There are many good programs, but very few
are well evaluated
Involve stakeholders
Community – University partnerships can be
very useful





Process as well as outcome evaluation
Quantitative and qualitative
Tailor evaluation to stakeholders and
situation
Evaluation is often undervalued
compared with “hard science”
Adequate resources should be devoted
to evaluation. 10% is recommended




Evaluation results should be used to
adjust programs and inform policy
Share results with stakeholders
Academic and practical training are
needed
The CDC Physical Activity Evaluation
Handbook provides a good framework

Gracias/Thank you