Inspire.
Engage.
Lead.
Getting Guidelines into Practice:
Lessons Learned from Research
Jenny Ploeg, RN, PhD
Evidence-Informed Decision Making Workshop
McMaster University
May 4, 2011
Outline
 Developing Guidelines
 Appraising Guidelines
 Implementing Guidelines
 Sustaining Guidelines
 Spreading Guidelines
Question
 What guidelines has your organization
implemented?
 What guidelines have you been involved in
implementing?
Practice Guideline Evaluation and Adaptation Cycle
(Graham et al, 2005)
Sources of Guidelines
 National Guideline Clearinghouse
 Registered Nurses Association of Ontario
 And many others
RNAO Guidelines Program
 Launched Nursing Best Practice Guidelines
Program with funding from MOHLTC in 1999
 42 published guidelines: English, French, Chinese,
Italian, Japanese, Spanish
 Guidelines for Personal Digital Assistants
 Educator’s Resource
 Toolkit for Implementation of Clinical Practice
Guidelines
RNAO Guidelines Program
 Best Practice Spotlight Organizations
 Champions Program
 Advanced Clinical/Practice Fellowships
 Long-Term Care Best Practices Initiative
 National Collaborative on Falls in LTC
 PhD Fellowships
Developing Guidelines
RNAO Development Panel
 Continence
 Constipation
Process of updating now, new evidence being
integrated
Levels of Evidence: RNAO
 Ia Evidence from meta-analysis or systematic
review of RCTs
 Ib Evidence from at least 1 RCT
 IIa Evidence from at least 1 well designed
controlled study without randomization
 IIb Evidence from at least 1 other type of welldesigned quasi-experimental study
 III Evidence from well-designed nonexperimental descriptive studies (e.g,
correlation studies, case studies)
 IV Evidence from expert committee reports or
opinions and/or clinical experience of respected
Appraising Guidelines
 AGREE II tool (Appraisal of Guidelines for
Research and Evaluation)
 Assesses methodological rigor and
transparency in which a guideline is
developed
Appraising Guidelines
23 items in 6 domains
 Domain 1: Scope and Purpose
 Domain 2: Stakeholder Involvement
 Domain 3: Rigour of Development
 Domain 4: Clarity of Presentation
 Domain 5: Applicability
 Domain 6: Editorial Independence
7-point scale: 1=strongly disagree, 7=strongly agree
Figure 1: The knowledge-to-action cycle
Harrison, M. B. et al. CMAJ 2010;182:E78-E84
Copyright ©2010 Canadian Medical Association or its licensors
RNAO Toolkit
 Toolkit for
implementation of
clinical practice
guidelines
Contents of Toolkit
 Stakeholder engagement (assessment form)
 Environmental readiness (assessment form)
 Evaluation of CPG implementation and impact (indicator
identification worksheet)
 Human and financial resources (Budget worksheet)
 Managing and monitoring implementation (action plan
template)
Implementing Guidelines
Edwards, Davies, Ploeg,
Dobbins, Skelly, Griffin,
Raphs-Thibodeau. (2005).
Evaluating best practice
guidelines. Canadian
Nurse, 101, 19-23.
Facilitators: Questions
From your experience, what are the three most
important factors that facilitate guideline
implementation?
Study: Facilitators and Barriers
 Design: Before-after study with quantitative
and qualitative data collection
 Setting: 22 agencies in Ontario that
implemented 7 RNAO best practice
guidelines from 2000-2001 (6-9 month
implementation period)
Ploeg, Davies, Edwards, Gifford & Elliott Miller. (2007). Factors influencing best
practice guideline implementation: Lessons learned from administrators, nursing
staff and project leaders. Worldviews on Evidence Based Nursing, 4, 210-219.
Study: Facilitators and Barriers
Guidelines:







Client Centered Care
Crisis Intervention
Healthy Adolescent Development
Pain Assessment
Pressure Ulcers
Supporting and Strengthening Families
Therapeutic Relationships
Study: Facilitators and Barriers
Data Collection:
 Semi-structured audio-taped telephone interviews
at end of guideline implementation
Participants:
 8 Clinical Resource Nurses (Implementation
leaders)
 58 Nurses and other care providers
 59 Administrators
Facilitators
Individual:
 Learning about the guideline
through small group
interaction
 Positive staff attitudes
Organizational:
 Leadership support
 Champions
 Teamwork and collaboration
Environmental:
 Professional
association support
 Inter-organizational
collaboration and
networks
Lessons Learned
 Attend to factors at multiple levels:
individual practitioner, social context,
organizational and environmental contexts
 Leadership support is key
 Champions
 Use of toolkits to help plan for
implementation
 Factors influencing implementation are
interlinked in complex ways not yet fully
Sustaining
Knowledge to Action
from: Graham et al: Lost in Knowledge Translation: Time for a Map?
Monitor
knowledge
use
Select, tailor
implement
interventions
Knowledge Creation
Evaluate
outcomes
Knowledge
inquiry
Assess barriers
to knowledge
use
Knowledge
synthesis
Knowledge
tools/
products
Adapt knowledge
to local
context
Identify Problem
Identify, Review
Select knowledge
http://www.jcehp.com/vol26/2601graham2006.pdf
Sustain
knowledge
use
Sustainability: Questions
 Has your organization been able to sustain
the guideline after initial implementation?
 What factors are most important for
sustainability of guidelines?
Why is sustainability planning rarely addressed in
research?
 Only 2 of 1000 sources screened for a literature review about the
diffusion of innovations in health service organisations included the
term sustainability (Greenhalgh et al., 2005)
 Shorter-term perspectives are the focus of health services research,
possibly due to:
– Limited availability of long-term funding
– Different time perspectives of policy cycles across health care
sectors
– Disengagement of project leaders towards the end of projects
Study: Determinants of the Sustained
Use of Research (SURE)
Investigators:
 Barbara Davies
 Nancy Edwards
 Jenny Ploeg
 Evangeline Danseco
 Tazim Virani
 Maureen Dobbins
Project Coordinator:
 Cindy Versteeg
Partners:
 RNAO
 Canadian Nurses Association
Funders: CIHR and CHSRF
Sustainability
 The degree to which an innovation continues
to be used after initial efforts to secure
adoption are completed (Rogers, 2003)
 When new ways of working become the norm
(Maher et al, 2007)
The Issue
 Up to 70% failure rate for organizational change and 30%
for health care change (Maher, et al. 2010)
 First 17 RNAO guidelines implemented from 2000-2004
 Six-month pilot implementation process, funded by
MOHLTC
 Once the initial pilot implementation funding is over, do
nurses, managers and senior executives continue to
implement guideline recommendations?
The Topics
Cycle 1
 Falls
 Continence
 Constipation
 Pressure ulcers (Assessment)
Cycle 2
 Healthy adolescents
 Client centered care
 Crisis intervention
 Pain
 Therapeutic relationships
 Pressure ulcers (management)
 Supporting families
Cycle 3
 Adult asthma control
 Breastfeeding
 Screening for delirium, dementia and depression
 Smoking cessation
 Reducing foot complications for people with diabetes
 Venous leg ulcers
Results
Participation rate:
90% Organizations (37/41) hospital, LTC, community
92% Decision-makers (112/122)
80% Nurses (77/96)
Sustainability Status at Year 2
Figure 1. Sustainability Status at Year 2, N=37
9
Number of Organizations
8
7
6
5
Not Sustained
Sustained
4
Sustained+Expanded
3
2
1
0
Cohort 1
Cohort 2
Cohort 3
Sustainability Status at Year 2
 43% Sustained (16/37);
Of the 16 sustained, 11 (30%) expanded
 57% Not sustained (21/37);
Most not sustained in the first and
second cohorts (81%)
Sustainability Status at Year 3
 59% sustained or sustained and expanded
(22/37)
 Organizations not sustaining at Year 2, likely
not to sustain at Year 3 (9/12 or 75%)
 Organizations sustaining and expanding at
Year 2, likely to sustain at Year 3 (10/11 or
91%)
Facilitators for Sustained or Expanded
Long-Term Use of Guidelines








Leadership by champions
Management support at all levels
Ongoing staff education
Guideline integrated into policies and procedures
Staff buy-in and ownership
Synergy with partners and external influences
Guideline characteristics
Multidisciplinary involvement
Implications
To ensure sustainability, we need:
 Ongoing and supportive leadership
 Management support
 Continuing education
 Organizational culture to support evidenceinformed practice
 Integration of guideline recommendations into
organizational policies, procedures, documentation
Implications
 Administrators and managers need to visibly support the
guideline through provision of resources and staff
education
 Staff education on guidelines needs to be ongoing
 Managers need to ensure there is funding for staff
education and time for staff to attend education sessions
during regular work hours
 Guideline recommendations can be integrated into staff
orientation and other professional development programs
SURE Study Conclusions
 Implementing changes in nursing practice to be more
evidence-based takes time, is dynamic, long-term and
iterative process
 Some organizations may take 2-3 years to show that
guideline recommendations are routine part of nursing
practice
 Full Report: CHSRF
http://chsrf.ca/final_research/ogc/pdf/davies_final_e.pdf
Spread
 Kudzu: vine that was introduced to the US from Japan in
1876
 One of fastest growing vines
 Now covers over 7 million acres in Southwestern US
 Grows as much as a foot per day in summer, climbs trees,
power poles, homes
 Can grow 60 feet per year
Research, Exchange and Impact for
System Support (REISS)
Co-Principal Investigators
Nancy Edwards, RN, PhD, FCAHS.
Professor, University of Ottawa
Doris Grinspun, RN, MSN, PhD, O.ONT.
Executive Director, Registered Nurses’ Association of Ontario
REISS Research Co-Investigators
Angela Downey
Associate Professor, University of
Victoria
Tazim Virani
Principal, Tazim Virani and
Associates
Ali Dastmalchian
Professor & Dean, Faculty of Business,
University of Victoria
Ian Graham
Associate Professor, University of
Ottawa
Irmajean Bajnok
Director, IABPG Program & CPNE, Registered
Nurses Association of Ontario
Ariella Lang
Research Scientist, VON Canada
Whitney Berta
Associate Professor, University of
Toronto
Heather McConnell
Associate Director, IABPG Program,
Registered Nurses Association of Ontario
Sheila Block
Registered Nurses Association of Ontario
Ann Lynch
Associate Director General of Clinical
Operations, MUHC
Barbara Davies
Professor, University of Ottawa
Patricia Marck
Associate Professor, University of
Alberta
Kathryn Higuchi
Associate Professor, University of
Ottawa
Cecile Michaud
Associate Professor, Université de
Sherbrooke, Campus de Longueil
Jenny Ploeg
Associate Professor, McMaster
University
Suresh Kalagnanam
Associate Professor, University of
Saskatchewan
Jennifer Skelly
Associate Professor, McMaster
University
Kim Jarvi
Senior Economist, Registered Nurses
Association of Ontario
Judith Ritchie
Associate Director for Nursing Research,
MUHC; Professor, School of Nursing
REISS: 5 Projects
1.
2.
Champions Promoting the Use of Best Practice Guidelines
Early Steps In Innovation: What Takes a Good Idea Further?
– early postpartum discharge
– minimal restraint use
– needle-exchange program
3. Spreading Innovation
4. Improving Communication to Improve Long Term Care
5. What Drives Cost and Enhances Benefits
Project summaries available: Nursing Best Practice Research Unit
Project 1: Best Practice Champions
Ploeg, J., Skelly, J., Rowan, M.,
Edwards, N., Davies, B.,
Grinspun, D., Bajnok, I.,
Downey, A. (2010). The role of
nursing best practice
champions in diffusing practice
guidelines: A mixed methods
study. Worldviews on
Evidence Based Nursing, 7,
238-251. doi:10.1111/j.17416787.2010.00202.x
Background
Definition of Champions: individuals who dedicate
themselves to supporting, marketing, and driving
through an innovation (Greenhalgh et al., 2005)
Variety of terms used: change agents, opinion
leaders, facilitators, linking agents, product
champions, best practice champions
Over 3,000 champions trained by RNAO since
1999
Background
“Knowing how champions contribute to projects is critical to
understanding, managing, and facilitating innovation and to
training others how to champion projects” (Markham,
1998)
 Little research on Nursing Best Practice Champions: roles,
activities and factors influencing their role
 Research evidence of impact is mixed, nature of role
remains unclear
 Most research conducted in acute care settings
Research Questions
 How do Nursing Best Practice Champions influence
the diffusion of BPG recommendations?
 What factors influence the role of Nursing Best
Practice Champions?
Design
Figure 1. Study Design: Mixed Methods Sequential Triangulation Design
Phase 1
Phase 2
QUAN
QUAL
QUAL
data
collection
QUAL
results
Develop
survey
instrument
QUAN data
collection
Interpretation
QUAL + QUAN
Note: QUAL: Qualitative; QUAN: Quantitative
QUAN
data
analysis
QUAN
results
Participants
Qualitative Interviews
23/26 (88.5%) Champions
Quantitative Survey
191/885 (21.6%) Champions
41/110 (37.3%) Administrators
Participants: Healthcare Sector
60
50
40
Cham I
Cham II
Admin II
30
20
10
0
Acute
Community
Long TC
Participants: Position
40
35
30
25
Cham I
Cham II
20
15
10
5
0
Front-Line
Educator
Manager
Findings
Champions:
 hold multidimensional roles: educator, facilitator, mentor,
leader, policy developer, evaluator
 are active knowledge disseminators of clinical information
to nurses
 work with various disciplines in all types and levels of
positions to explain, convince and help ensure guideline
implementation
Findings
Champions:
 use many strategies at multiple organizational
levels
 attend to various stakeholder groups
 tailor diffusion strategies to organizational context
Diffusion Strategies of Champions
1.
Dissemination of information about clinical practice guidelines
– Education and awareness
– Acting as a resource to support and mentor nurses
2. Champions as persuasive practice leaders
– Working through committees
– Participating in and leading interdisciplinary teams
3. Tailoring the guideline implementation strategies to the
organizational context
– Exploring, auditing, monitoring of best practices
– Documentation changes to incorporate best practice
recommendations
Spread Within Organization by Setting
60
50
40
30
20
10
0
Hospital
Community
Yes
LTC
Other
No
Spread Beyond Organization by Setting
30
25
20
15
10
5
0
Hospital
Community
Yes
LTC
Other
No
Factors Associated with Spread:
Logistic Regression Analysis
 Spread from a unit/team
within an organization was
dependent on the success of
the implementation
strategies
 For every 1 unit ↑ in success
of implementation you are
2.0 times as likely to have a
BPG spread beyond a unit or
team
 Spread outside the
organization was
dependent on the extent of
change in team unit
practices
 For every 1 unit ↑ in extent
of change you are 6.8 times
as likely to have a BPG
spread beyond the
organization
Discussion
 Champions hold multiple roles:
facilitator, change agent, knowledge translator
 Champions are persuasive practice leaders, work with
many disciplines
 Champions work through complex web of committees and
working groups
 Champions are adaptors who tailor BPG implementation
strategies to organizational context
Implications
 Adequate and ongoing training to maximize potential of
champions role and impact
 Broad range of knowledge and skills: knowledge transfer,
policy development, research and evaluation, leadership,
mentorship
Future Research
 What criteria should be used to select champions?
 Are characteristics of champions associated with
effectiveness? (e.g., position, credibility, experience)
 Examine power, span of control, authority of champions
 What organizational supports are required for success of
champions’ role?
 What constitutes a ‘critical mass’ of champions?
 What is the impact of champions?
Study 3: Spreading Innovation –
The Best Routes to Best Practices
Barbara Davies, Judith Ritchie and team
What We Studied:
Examined naturally occurring diffusion of guidelines how practice guidelines spread within and between
organizations under normal, everyday conditions.
Phase 1: We created diagrams to map how innovations
spread within and outside an organization, two years after
implementation of an RNAO BPG.
Phase 2: We then visited 2 sites 7-10 years after original
introduction.
Results
Phase 1:
 Spread diagrams revealed the dynamic and
non-linear processes occurring in organizations
as they adaptively facilitated the spread of a
guideline.
 Most sites, except for a community care site,
had a higher level of internal organizational)
spread than external spread.
Phase 2 Hospital (Acute)
 Initially funding was allocated to guideline
implementation but currently “in-kind”
implementation is ongoing
 Guideline implementation of the index topic
(supporting families ) continues to be sustained 10
years later
Phase 2 Community Home Visiting
 Guideline implementation of the index topic (venous leg
ulcers) continues to be sustained 7 years later
 Challenges, some of the guideline-based requests to the
case managers at the funders (CCACs) are not supported.
 The spread diagram was modified to reflect the central and
multi-faceted “political funding support/interference
process” of the CCACs in a diamond shape box.
Lessons Learned
Takes a combination of strategies: learning opportunities,
champions, discussion, communicating goals
Frontline workers “deeply involved” to figure out barriers
and ways around them
Strong leadership at every level to align vision, goals +
activities
RNAO guidelines were sustained 7 to 10 years later
“Absolutely better” to be a patient at the hospital now than
10years earlier
Implications
Sustaining evidence-informed innovations for the
long term depends on full engagement of leaders
and staff in their implementation.
When people feel involved, they will see its value
and benefits and stick with it in the long term.
Overall Program of Research: Model
Guideline Implementation for Improved
Client/patient Outcomes:
A longitudinal multi-site study
GICOM
Academic Investigators
1.Nursing, uOttawa: Barbara Davies, Kathryn Higuchi, Mary
Ann Murray, Jackie Ellis
2.Nursing, McMaster University: Sandra Ireland, Jenny Ploeg
3.Clinical Epidemiology Unit, Ottawa Hospital Research
Institute: Monica Taljaard
BPSO Candidates -2009-2012
(16)
 Formal three year
partnership with RNAO
 Focus on enhancing their
evidence based nursing
practice cultures, and
quality outcomes.
 RNAO continues to work
with the BPSOs to create
strategies for sustainability
and system-wide spread.
GICOM settings (9)
Acute Teaching Hospital: 2
Community Hospital: 2
Community health centre: 1
Home care Nursing: 2
Long-term care: 1
Rehabilitation care: 1
NHS sustainability model:
A promising developing
model
 Initiated because sustainability was a problem
 Aim to identify key factors which support or hinder
sustainability of improvement initiatives
 Create a model that practitioners could use to
assess the likelihood of sustainability
 Using a Bayesian approach 250 experts identified
measures, created factors and estimated the
diagnostic power of the factor levels
 Reported as easy to use
 Model being used in England, USA, Sweden and
Norway
Sustainability Model
(Maher L, Gustafson D, Evans A. NHS Institute for Innovation and
Improvement 2010)
Monitoring progress
Training and involvement
Behaviors
Adaptability
Staff
Credibility of benefits
Senior
leaders
Process
Clinical leaders
Organization
Benefits beyond helping patients
Fit with goals and culture
Infrastructure
www.institute.nhs.uk/sustainability
Outcomes
 Need for more focus on outcomes of implementing
and sustaining and spreading guidelines at multiple
levels
Implementation of a Falls Prevention Guideline Across the
Hamilton Niagara Haldimand Brant Local Health Integration
Network: A Longitudinal Study
Jenny Ploeg, McMaster University
Sandra Ireland, Hamilton Health Sciences
Barbara Davies, University of Ottawa
Kathy Higuchi, University of Ottawa
Mary Ann Murray, University of Ottawa
Karen Cziraki, Hamilton Niagara Haldimand
Brant Local Health Integration Network
Nancy Fram, Hamilton Health Sciences
Helen Kirkpatrick, St. Joseph’s Healthcare
Hamilton
Aleksandra Zecevic, University of Western
Ontario
Research Questions
1. What are the fall rates and total number of serious falls at
three community hospitals providing acute care services
within the LHIN before, during and after a mentored
implementation of the RNAO Prevention of Falls and Fall
Injuries in the Older Adult Best Practice Guideline?
2. What is the projected economic benefit to the LHIN of
reducing serious falls (i.e., those resulting in injury) at
three community hospitals?
Research Questions
3. What are the experiences of project leaders and decisionmakers in community hospitals related to implementing
and sustaining a falls prevention guideline? (e.g.,
strategies used to assist nurses and other health
professionals to achieve improvement in fall outcomes;
lessons learned in providing feedback related to fall
outcomes)
4. What barriers and supports are encountered when
implementing and sustaining strategies to improve patient
fall-related outcomes over time?
Sustainability Scores for Site 1
Sustainability Score Bar Chart
0
2
4
6
10
8
Benefits
Credibility of the evidence
Adaptability
Monitoring progress
Involvement and training
Behaviours
Senior leaders
Clinical leaders
Fit with goals and culture
Infrastructure
Score
Max
12
14
16
Challenges
 Agreement on common definition of a serious
fall
 Difficulty experienced by participants in using
the NHS survey
 Competing pressures and limited resources
within sites
 Variable stages of implementation of the Falls
BPG at baseline
For details and other publications visit
 www.nbpru.ca
 www.rnao.org
Thank you: Questions?
Contact:
Dr. Jenny Ploeg
Email: ploegj@mcmaster.ca
Dr. Ploeg is a MOHLTC mid career nursing research
award recipient