The Implementation Process:
Perspectives from Frontline
Providers and Managers
JoAnn E. Kirchner, MD
Louise E. Parker, PhD
Laura Bonner, PhD
Elizabeth M. Yano, PhD, MSPH
Mona Ritchie, MSW
Agenda
 Formative Evaluation
– TIDES
 Methods
 Findings
Formative Evaluation
 Types of formative evaluation
– Developmental evaluation
– Implementation-focused evaluation
– Progress-focused evaluation
– Interpretive evaluation
 The “process” as a product
Formative Evaluation: TIDES
 Developmental evaluation
– Pre-implementation stakeholder interviews with MH and
PC leadership
 Implementation-focused evaluation
– Mid-implementation qualitative interviews
– Early feedback of initial findings to implementation team
 Progress-focused evaluation
– Program penetration and utilization
 Interpretive evaluation
– Site case studies
– Cost analysis
– Qualitative interviews
Methods
 Conducted 106 qualitative semi-structured
interviews during field visits with
– Managers across four participating medical
centers and three participating regional
networks
– Providers, administrators, and consumers
across five participating facilities
 Analysis
– Data management package
– Coded by two-investigator teams
– Independent coding with resolution of conflicts
Methods
 Here we report findings from
– 19 medical center and regional network
managers
– 49 Frontline providers and administrators
 Concerning their perspectives on the QI
program implementation process
Results
Local Site
Resources
Local Frontline Staff:
Attitudes
Behaviors
Implementation
process
Penetration and
Sustainability
Local Outcomes
Spread
Importance of site characteristics
Attitudes/beliefs/experiences
–
–
–
–
Perceived need for the intervention
Competing needs
Staff open to innovation
MH/PC relationship
• Expected would affect implementation
success but did not
Importance of site characteristics
Resources
– Perceived time to
• Utilize the program
 Marketing opportunity
• Participate in implementation activities
– Organizational structure
– Staffing
– Prior QI experience
– Tools (e.g., access to informatics support)
Who to Involve
All Informants
– Medical Center Managers
– Frontline Clinical Managers
– Frontline Primary Care Providers (MD, PA, APN)
Frontline Providers
– RNs, LPNs
– Mental Health Providers
Key Roles in Implementation
 Leadership
leadership support is obviously also critical. Unless the leaders are able to
translate the importance of the initiative, it's probably not going to take
place for a variety of reasons. One of which is that there are a thousand
things that need to be done, priorities need to be set and it’s leaderships
responsibility to make sure that those priorities are delineated and made
clear to everybody, what is it that we’re going to do and what is it we're not
going to do - Chief of Staff
– Importance of leaders
• valuing the program
I think it’s [leadership support] important because if the leadership
shows that it’s important enough for them to spend their time,
then it implies that it’s important enough for us to spend our time MH Provider, Physician
Key Roles in Implementation
• initiating the program
• “cheerleading”
Managers just have to go back out and continue doing cheerleading
sessions. A little more cheerleading. You go back, and you ... when you
see that that’s where you are, you have to just go back and reemphasize.
Okay. We’ve gotten to here, we’ve got to be over here or...you guys are
doing a great job here, but we could do a little more here, and it’s that
cheerleading. You know, 90 percent I think of what managers do are
cheerleading people on - Network Manager
• resources
I feel that anything that you have in place, if the head of anything...is not
going to support you, give you what you need, be there for you in every
sense of the word, it’s not worth having. I think it is doomed to fail if you
don’t have that in place - Network Manager
• importance of management at multiple levels
Key Roles in Implementation
 But in absence of clinic perceived need for the
program, leadership will have an uphill battle
Key Roles in Implementation
 Champion
– Who should be a champion?
Does that person have credibility with the other staff, you know, and
that’s not measured in what letters you have behind your name. That’s
do people listen to you when you speak. I don’t know how you put a
degree on that, but that’s what you need - Primary Care, RN
 Colleague who has experienced the program
 High status peers
 Someone interested in quality improvement and in
improving mental health care in primary care setting
 Credible
 Able to communicate well to groups
 Self-driven
 Good rapport
Key Roles in Implementation
 Role/activities of the champion
He’s [the local champion] been the person who’s at our meetings reminding
us about the program…And I think the more you do that, the more you get
people using it - Primary Care, RN
–
–
–
–
Serve as the local expert and resource person
Encourage people to attend related meetings
Remind people about the program
Provide ongoing marketing
 Needs of the champion
– Time
Key Roles in Implementation
 Leadership as a champion
– Roles of champion, leadership, and opinion
leaders are not distinct in the views of clinical
staff and managers
– Thus, implementation researchers may be
creating artificial distinctions
Site Readiness
Low hanging fruit
What about the fruit higher in the tree?
Informing Readiness
 Message will depend on how ready sites are
for a particular intervention (i.e., perceived
need)
 Regardless of readiness, different audiences
(i.e., managers versus frontline) have different
needs to promote readiness
 Importance of ongoing marketing for all
Sometimes in the VA, you’ve got to tell me three times to do it because if
you tell me three times, it was really important. If you tell me once and
I never hear from you again, then it probably wasn’t important and was
a passing fad and I’ll kind of wait – Chief of Staff
PLAN
2nd Generation Sites
ACT
DO
1st Generation Sites
• sustainability
• penetration
1st Generation Sites
STUDY
Ongoing Formative Evaluation
 Developmental evaluation
– Site level needs assessment tool
 Implementation-focused evaluation
– Site level fidelity monitoring tool
 Progress-focused evaluation
– Program penetration and utilization through National
Depression Monitor
 Interpretive evaluation
– Formative evaluation of regional dissemination of TIDES
– Assessment of the sustainability effort