I want to implement a new program, How do I so successfully with providers?
Conducting Implementation Studies
AAWC WocSpec Research Committee, Barbara M. Bates-Jensen, PhD, RN, FAAN
ABSTRACT
PURPOSE: To provide information on how to conduct implementation studies.
BACKGROUND: Research methodology is concerned with how a study’s
design is implemented and how the research is conducted. Methodology
determines the quality of the data generated by the study and specifies: when
and how often data are collected, construction of data collection measures, who
data is collected from, how data is analyzed, and how findings are presented.
Study design is a part of research methodology but not the only part.
METHODS: Implementation science uses a variety of approaches and designs
for implementation studies typically include one group pre-test/post-test design.
While readers may be familiar with the design, implementation science is less
well understood. Implementation studies is the examination of the actual
process used to try to get providers to change behavior and practice. These
studies are often poorly conducted simply because of a lack of understanding of
what implementation research actually entails. We present information on
conducting implementation studies using a basic group pre-test/post-test
design. Specific implementation interventions are presented and typical
missteps outlined.
CONCLUSIONS: Implementation research studies involve the examination of
the process of how an investigator changes provider behavior and practice.
Strategies for implementation include use of : cheerleader or thought leaders,
printed materials, reminders, lecture/education sessions, identification of
barriers and obstacles, and general and individual audit and feedback. Critical
elements of implementation studies include allowing sufficient time to evaluate
uptake of the new program or behavior and determining intensity of the
behavior. Usually a single group is studied but no comparison between an
equivalent control group is made, rather the comparison is often retrospective..
the scientific study of methods to promote the uptake of research findings..
BACKGROUND
Implementation Research is concerned with
examination of methods used to change provider
behavior and practice. Specific designs and
strategies are employed to change how providers
deliver care.
Often research projects focus on small scale pilot
studies or laboratory based experiments, and
assume that findings can be generalised to roll out
into a practice based domain with few changes.
Implementation research explores the challenges
that are faced when generalising research findings
'in the real world‘. Concerns relate to:
• How innovations are diffused into real world
environments
• Data collection measures & instruments are key
• Generally best if built on a theoretical framework
for translating science into practice
In Implementation studies, the data are concerned
with the “how” a program or intervention is
implemented into practice.
Whether or not the study results can be expected
to be the same with other programs that are
implemented with other groups of people (external
validity) are issues that are addressed by
methodology. mp
Stages of Implementation
Implementation studies can examine any of the
stages of implementation:
Exploration and Adoption—studies may focus on:
identifying the need for an intervention
acquiring information via interactions
assessing the fit between the intervention
program and organization needs
Program Installation—studies focus on:
resources needed to get started for actually
doing things differently
structural supports necessary for initiation
ensuring the availability of funding streams,
human resource strategies, and policy
development
Initial Implementation—studies focus on:
changes in skill levels
organizational capacity
organizational culture
education, practice, and time required for new
behaviors to mature.
Full Operation—studies focus on:
full staffing, full client loads, and all of the
realities of “doing business” with the newly
implemented evidence-based program.
outcomes from new practice
evaluates new practice based on fidelity
measures reaching a certain criterion.
Innovation—studies focus on:
attempted implementation of evidence-based
practices and programs presents an opportunity
to learn more about the program itself
Understand the conditions under which the
innovation can be used with fidelity and good
effect
discriminate between drift and innovation
Sustainability—studies focus on:
Maintaining the new behavior/program over the
long term
Retraining, dealing with turnover.
Some considerations to improve
Implementation studies
Key data to include in implementation studies—
strategies for implementing the change:
What strategies were done?
Education
Clinical reminders
Audits
Feedback
Cheerleaders or thought leaders
Printed materials
Identification of barriers and obstacles
Collaboratives
How much time was spent on each activity?
How many staff attended sessions?
Who conducted sessions? Education of leader?
Measure whether or not the education actually results
in change in behavior. No one cares if a posttest
shows more correct answers, this does not translate
into using the knowledge to improve care delivery.
Present data on the costs associated with the
educational program: time, staff resources to present
the education, technology required., etc.
Present data on attendance at the program (100%
attendance? 75%?)
Key data to include in implementation studies—
who were the providers?
Who are the key providers whose behavior was
changed?
Education level
Professional level
How to make Implementation studies
stronger:
Use an implementation science theoretical model to
frame the study
Measure fidelity of the intervention/program
How much of the innovation is actually
implemented?
•75%? 90%?
Measure the drift of the innovation/program
How far from the original program has the
implemented program drifted? Usually a program or
intervention is initially implemented at a higher (or
closer to the plan) level and over time the
intervention “drifts” back towards the original
behavior
Measure the innovation of the program
Most programs/interventions have to be changed
slightly to work in a particular organization,
describe and specify how the intervention was
changed for a particular organization
Include how long the intervention has been
maintained
Outcomes for implementation studies relate to
PROCESS…how the intervention is being practiced
How to measure the intervention
Key data to include in implementation studies—
how long is the follow up period?
How will you determine whether or not the
behavior/new program was sustained?
Must have some follow along measurement periods to
establish that the new behavior/program is maintained
into practice
Use research staff to complete the same behaviors at
100% and then compare to staff measures
Use audit data to determine how much of the new
intervention/program are actually implemented