Developing and Managing Quality
Improvement Learning Collaboratives
Lessons from the MLC States
September, 2010
Contributors:
Brenda Joly PhD, MPH
Maureen Booth, MRP
George Shaler, MPH
Ann Conway, PhD
Overview
• Evaluation background
– Components
– Case study process, rationale, theory
• Findings and opportunities
– Planning and start-up
– Managing the Learning Collaborative
• Implications
– Sponsors
– Local health departments
MLC Evaluation
• Quality improvement goal:
– To promote the application of QI methods
• Evaluation tools:
– Annual survey
– Mini-Collaborative survey
– Quarterly reports
– Case studies
– Key informant interviews
Case Studies
• Who?
– Mini-collaborative members and organizers
– LHD quality improvement teams
• Why?
– Multiple perspectives
– “On-the-ground” understanding
• What?
– Site visits, interviews, document review
– Observations of meetings
Literature-Based Theory
Cohesive
Team
Sufficient
Time
Productive
Meetings
Strong
Facilitators &
Content Experts
Clear
Expectations
Case Study Theory
Resources
Provided to
Agencies
Sufficient
Training and TA
Successful MiniCollaboratives
Member
Motivation and
Engagement
Work Plan in
Place and Used
Use of an
Improvement
Model
Senior Leader
Commitment
Available
Evidence and
Best Practices
Planning and Start-Up
Key Finding #1:
The Relevance of a Target Area is
Critical and Impacts Engagement
• Who selected targets?
– State sponsors or statewide body
– Local input
• What criteria were use in selection?
– Relevance and alignment with priorities
– Need demonstrated by data
Key Finding #2:
The Structure of a Mini-Collaborative
Affects its Effectiveness
• What works best?
– Involving senior leadership is essential
– Having diversity in faculty adds value
– Having a sufficient size to promote exchange
– Working on a consistent QI project
– Having clear roles and responsibilities
– Having prior experience working together
Key Finding #3:
Defining Expectations & Communicating
Them in Advance Builds Confidence
• What did we learn?
– Goals of mini-collaboratives varied and evolved
• Build QI skills
• Improve quality within target area
– Goals should be aligned with timeframe and
participant readiness
– Expectations for QI projects should be focused,
realistic and communicated
Key Finding #4:
Advanced Planning Influences the
Effectiveness of Learning Collaboratives
• Who planned the mini-collaboratives?
– State sponsors
– Broader group of state, faculty and others
• What should be planned?
– Model, curriculum, workplan, use of faculty and
evidence, tools to assess participant knowledge
and measures to monitor progress
Key Finding #5:
The Level of Effort Among Sponsors
and LHDs is Often Underestimated
• What did we learn?
– The planning and start-up phase is especially
time consuming for sponsors
– Having outside faculty helps
– Level of effort is often underestimated if
expectations are unclear or scope is unrealistic
– Competing priorities are a reality
Managing the Learning
Collaborative
Key Finding #1:
Opportunities for Timely and Frequent
Application of QI Tools are Essential
• What was most helpful?
– Learning/using fewer tools
– Training on when and how to use
– Providing opportunity for immediate application
– Providing feedback on use
– Giving LHDs a chance to practice, practice,
practice
Key Finding #2:
Communication With and Among
Members Enhances Learning Experience
• What was most helpful?
– Opportunities for ongoing information exchange
– In-person meetings
– Structured learning sessions
– Communicating the value of QI to LHDs
– Site visits to LHDs
– Communicating clear expectations
Key Finding #3:
Several Factors May Strengthen or
Impede a QI Learning Collaborative
• Facilitating factors?
– Commitment to data driven QI and accreditation
– Leadership buy-in
• Impeding factors?
– Concurrent QI skill development at state and
local level
– Lack of focus and clear expectations, unrealistic
goals, turnover, and competing priorities
Key Finding #4:
It is Critical for Participants to Have
Access to Adequate Resources
• Technical assistance:
– There should be a good mix of content expertise
and QI expertise among sponsors and faculty
• Other resources:
– Evidence needed to select QI intervention should
be made available
– Financial resources may boost efforts of LHDs
Implications for Sponsors
• Some considerations…
– Set time aside for advance planning
– Recruit credible and skilled faculty
– Assess QI knowledge beforehand
– Recruit the enough and the right mix of
participants
– Embed the application of QI into the process
– Communicate with the group frequently
– Follow a structured approach
Implications for LHDs
• Some considerations…
– Garner support of leadership and staff
– Promote the value of QI
– Align QI project with job responsibilities
– Actively engage in information exchange
– Use faculty and available technical assistance
– Remain open to the process and outcomes
– Document your efforts
– Share your findings
Developing and Managing a Quality
Improvement Learning Collaborative
Lessons from Florida
September 2010
Contributors:
Cathy Montgomery, M.S., ASQ-CQIA
Sandra Ruzycki, M.P.H., Quad-R
Baker, Clay, DeSoto, Duval, Glades, Martin, Nassau, St.
Johns and St. Lucie County Health Departments
Planning and Start-Up
Plan, Plan, Plan
• Use data to determine target
area
– Overweight and obesity
among children 6-19
• Create partnerships
• Outline timeframes and
expectations Ψ
• Establish criteria for
selecting participants
Ψ = lesson learned
Use Criteria
Childhood Obesity Mini-Collaborative
Establish (facilitator) Goals
• Use of quality
improvement tools and
methods
• Support participants in
their projects
• Communicate project
impacts and outcomes
– Local, state, national
venues
Select an Improvement Model
• Quality Improvement Control
Story Ψ
– Founded on PDCA
• Integrates QI tools and
methods throughout the
process
Training and Tools
• Assess participant’s
knowledge of QI tools Ψ
• Provide relevant training
– Just-in-time
– Face-to-face Ψ
• Allow participants to
practice using tools and
process
Results: Training
• Participation in QI training improved overall
knowledge of QI tools – 84%
– Baseline = 84%
• Confident in their ability to use QI tools – 68%
– Baseline = 62%
• % of participants that rated level of use of QI tools
after training as:
moderate – 58%
– Baseline = 62%
high – 21%
– Baseline = 15%
*Surveys conducted March 2009 (baseline) and February 2010
Results: Tools
% of participants that indicated a high readiness to:
• Use QI tools in other initiatives – 83% (n = 15)
• Incorporate QI tools into ongoing work – 67% (n=12)
• Development of evaluation measures – 56% (n = 10)
• Present QI tools to CHD staff – 50% (n=8) Ψ
Technical Assistance
• Conduct coaching calls
–
–
–
–
Using quality improvement process and tools
Creating action plans and storyboards
Developing methods and measures for evaluation
Analyzing data results
• Resources
– Subject matter expertise
– Books, articles, and research related to QI and projects
– Evaluation database
Evaluation Database
Managing the Learning
Collaborative
Collaborative Teams
• Counties created teams
consisting of internal
staff
– 65% of participants
indicated multiple CHD
programs were
represented on team
– 3 of 9 counties had QI
program staff on the team
Results: Collaborative Teams
• Team meetings were random and unproductive Ψ
– 53% of participants stated team meetings were only
slightly productive
– 53% moderately or slightly agreed there was consensus
on project goals and procedures
• % of participants that strongly or moderately
agreed project team members:
– Got along well – 94%
– Treated each other fairly – 71%
– Had necessary skills for this project – 65%
Results: CHD Support
• 59% rated level of CHD
support was very or
somewhat supportive
• 71% indicated staff time
and resources were fully,
mostly or somewhat
available
Communication
• Provide venues for participants to share ideas
– Monthly conference calls/web ex
– Face-to-face meetings
– SharePoint site
• Conduct one-on-one coaching calls
• Communicate progress of collaborative projects Ψ
Results: Communication
• % of participants who rated the following resources
as useful
–
–
–
–
–
One-on-one coaching calls: 65%
Monthly conference calls: 65%
SharePoint site: 59%
Collaborating with other MCLC counties: 71%
Collaborating with other agencies: 53% Ψ
• % of participants who agreed or strongly agreed HPI
provided:
– Effective communication: 94%
– Effective project coordination: 95%
Overall Experience
• 76% rated their overall experience
as excellent or good
• 94% rated the level of success of
their project as excellent or good
• 88% indicated they were likely to
participate on another learning
collaborative
• 76% reported they were likely to
implement a new QI project in
their CHD
Lessons Learned Ψ
What else did we learn?
• Planning and implementation is time consuming
– 41.2% of participants spend 1-5 hours per week on
project; others indicated they spend more time!
• Don’t assume what they know
• Competing priorities are a reality