Leadership Accountability
Demonstration Project
Intensive group Horizontal Learning Call1
Pre-project safety culture survey debrief
Sep. 15th, 2014
© The Johns Hopkins University, The Johns Hopkins Hospital, and Johns Hopkins Health System
LADP Project Roadmap
 June –August 2014
• Site visit conversations
 August 2014
• Cohort tool webinar
 Try out the Aligning Goals Across the
Organization tool
 Try out the Defining Safety Competencies
tool
• Individual site coaching calls begin
 September 2014
• Cohort horizontal learning webinar
• Cohort tool webinar
 Try out the Strategic Risk Assessment and
Prioritization Tool
 Try out the
 Strategic Learning from Defects Tracer Too
• Individual site coaching calls
 October 2014
• Cohort horizontal learning webinar
• Individual site coaching calls
 November 2014
• Cohort tool webinar
• Try out the Collaborative Tactical Decision
Making Method
• Interactive business case presentation
• Individual site coaching calls
 December 2014
• Cohort horizontal learning webinar
• Individual site coaching calls
Roadmap for cohort horizontal learning calls
Goals:
• Connect cohort 2 teams
• Share, teach, and learn
• Reflect and innovate
Today will be a bit different given we are just getting
started
1. I have data….but now what?
2. Some food for thought regarding next steps and
debriefing survey results
Overview
• Section I
Summary of Overall LADP Cohort 2 HSOPS Results
• Section II
HSOPS Debrief (Tool)
• Section III
Summary of Overall LADP Cohort 2 PSOA Results
• Section IV
PSOA Debrief (Tool)
• Discussion and Next Steps
4
Organizational culture and strategy are 2
sides of the same coin
Image: http://www.rootinc.com/blog
Section I
Summary of Overall LADP Cohort 2
Hospital Survey on Patient Safety (HSOPS) Results
What is Safety Culture?
 Perceived priority of safety relative to
other goals
 Culture is the compass team members
use to guide their behaviors, attitudes,
& perceptions on the job
• What will I get praised for?
• What will I get reprimanded for?
• What is the “right” thing to do?
Culture provides the context
for team success
Image source: Marysia Tomaszewska, August 8, 2012, used
under a Creative Commons License
7
Why Safety Culture Matters
1.
Safety culture is related to outcomes
• Patient outcomes




Patient care experience
Infection rates, sepsis
Postop. hemorrhage, respiratory failure, accidental puncture/laceration
Treatment errors
• Clinician outcomes

2.
Incident reporting, burnout, turnover
Safety culture influences the effectiveness of other safety and quality
interventions
• Can enhance or inhibit effects of other interventions
3.
Safety culture can change through intervention
• Best evidence so far for culture interventions that use multiple
components
Huang et al., 2010; Mardon et al., 2010; MacDavitt et al.,
8
2007; Singer et al., 2009; Sorra et al., 2012; Weaver, 2011.
HSOPS Background
• The Hospital Survey on Patient Safety Culture was sponsored by the
Quality Interagency Coordination Task Force (QuIC), a group
established in accordance with a 1998 Presidential directive to ensure
that all Federal agencies involved in purchasing, providing, studying, or
regulating health care services are working together and toward a
common goal of improving quality care. The survey was funded by the
Agency for Healthcare Research and Quality (AHRQ).
• The development of this safety culture assessment tool included a
review of the scientific literature pertaining to safety, error and
accidents, as well as error reporting. In addition, hospital employees
and managers were interviewed to identify key patient safety and error
reporting issues. Other published and unpublished safety culture
assessment tools also were examined.
Overall Picture
VHA HSOPS Composite Scores (Dimension Scores)
N = 228 clinicians and staff (43.05% response rate) (as of 8/30/14)
80%
85%
Unit-referenced dimensions
Teamwork Within Units
73%
76%
75%
74%
Organizational Learning—Continuous Improvement
Supervisor/Manager Expectations & Actions Promoting…
Feedback & Communication About Error
Communication Openness
Staffing
46%
44%
39%
Outcomes
Hospitalreferenced
dimensions
Nonpunitive Response to Errors
Management Support for Patient Safety
65%
66%
62%
59%
57%
62%
59%
58%
Teamwork Across Units
46%
40%
Handoffs & Transitions
Frequency of Events Reported
Overall Perceptions of Patient Safety
55%
0%
AHRQ 2012
73%
10%
20%
Percent Positive (%)
30%
40%
50%
60%
64%
67%
67%
70%
80%
90%
Remember: Culture is Local
2 Work Areas in Same Hospital
96%
94%
Unit-referenced dimensions
Teamwork Within Units
81%
Organizational Learning—Continuous Improvement
81%
78%
Supervisor/Manager Expectations & Actions Promoting…
68%
Feedback & Communication About Error
Communication Openness
62%
Staffing
Hospitalreferenced
dimensions
Outcomes
Frequency of Events Reported
71%
48%
42%
69%
81%
64%
63%
Teamwork Across Units
Handoffs & Transitions
46%
56%
58%
85%
64%
67%
Overall Perceptions of Patient Safety
0%
Unit 2
87%
70%
34%
Nonpunitive Response to Errors
Management Support for Patient Safety
94%
20%
Unit 1
40%
60%
80%
100%
Understanding the HSOPS Aggregate Report
Who completed the survey: Pg. 4-6
Composite Score: Pg. 7-8
Scores = Percent positive responses
Composite Score: Pg. 7-8
Scores = Percent positive responses
Interpreting Composite Scores:
• The big picture view
• Higher is better
Questions: Pg. 9-26
Percent positive = Green; Percent neutral = Yellow; Percent negative = Red
Item charts provide a deeper dive:
For positively worded items, more green is better
A6. We are actively
doing things to
improve patient
safety.
Current
Other Networks
31
95
4 5
91
*For negatively worded items, more RED is better
A8. *Staff feel like their
mistakes are held
against them.
16
59
30
28
25
42
Section II
Using the CUSP Culture Check-up Tool to Debrief Your
Hospital Survey on Patient Safety Culture (HSOPS)
Results
Next Steps: Meaningfully using the HSOPS
results for your participating units
• Debriefing is…
• A semi-structured conversation among frontline clinicians
and staff that is usually led by a designated facilitator
• Purpose…
1. Encourage open communication, transparency, and
interactive discussion about the survey results
•
Across all levels
2. To engage clinicians and staff in generating and
implementing their ideas about how to create an
effective safety culture in their work area
CUSP Culture Check-Up Tool
• What is the Purpose of this Tool?
• Understand the culture of the unit
• Use teammates’ feedback to predict barriers to change and avoid them
• Use feedback to make the most of the team’s strengths
• Who Should Use this Tool?
• Safety culture debriefing facilitators
• Use this tool to help guide the discussion and record group decisions
• Where can I Find this Tool?
• https://armstrongresearch.hopkinsmedicine.org/vha.aspx
CUSP Culture Check-Up Tool
How Do I Use this Tool?
• Share culture results with everyone on the unit
• Bring together team members from your work area
• Follow your debriefing plan
• Take notes and recognize recurring themes
• Purpose = Open, honest discussion about ideas to make the culture of your
work area the best it can be
• Focus on identifying system issues that the group can work on improving
together instead of individuals
• NOT used to point fingers at specific individuals
• Use the tool to structure meetings and guide conversation
• As a group, complete all steps in this worksheet
Steps in CUSP Culture Check-Up Tool
• STEP 1: Your team identifies the general strengths and weaknesses of your unit culture.
• STEP 2: Your team identifies the specific behaviors and attitudes that make up those strengths and
weaknesses.
• STEP 3: Debriefing facilitator encourages group reflection. Your team chooses opportunities for growth,
understanding that cultural strengths can help fix cultural weaknesses.
• STEP 4: Your team identifies a strategy for fixing the opportunities selected in step three.
• AHRQ recommends creating ‘safety briefings’ – short updates for frontline teammates about patient
safety issues in the work are. For more ideas, go to:
http://www.ahrq.gov/qual/patientsafetyculture/hospimpdim.htm.
• STEP 5: Your team works out the details of putting strategy into action.
• STEP 6: Your team evaluates your plans. Be sure to meet again and check in on progress at your SUSP team
meetings
The “Culture Check Up Tool” = Word Document that
Debriefing Facilitator can use to guide conversation &
improvement planning
In Sum
1. Review the survey report for your unit
2. Can be helpful to distill the report down into 3-5 key slides
3. Decide when, how, and where to debrief your teammates (and
leaders) on these results
• Be prepared to listen
• Ask for feedback
• Ask teammates to help come up with solutions
4. Gather a small group together and use the “culture debriefing
tool” to examine the roots of problem areas and begin to
formulate strategies for improvement
Section III
Summary of Overall LADP Cohort 2
Patient Safety Organizational Assessment (PSOA)
Results
PSOA Background
The VHA PSOA Tool:
utilizes categories defined by the Malcolm
Baldrige National Quality program as the
framework
identifies the status of critical safety functions,
called “Key Aspects of Safety”
provides a systematic method to evaluate current
processes and systems
measures ongoing progress in establishing a safer
organization
PSOA Domain: Key Aspects of Safety (KAS)
1.
Leadership
• (L1) Demonstrate patient safety as a top leadership priority
• (L2) Promote a non-punitive culture for sharing information and
lessons learned
2.
Strategic Planning
• (SP1)Routinely conduct an organization-wide assessment of the risk of
error and adverse events in care delivery processes
• (SP2) The organization actively evaluates the
competitive/collaborative environment and identifies partners with
whom to learn and share best practices in clinical care:
3.
Measurement, Analysis and Knowledge Management
• (MAKM) Analyze adverse events and identify themes across events
28
PSOA Domain: Key Aspects of Safety (KAS) Cont.
4. Workforce Focus
• (WF1) Establish rewards and recognition for reporting errors and
safety driven decision-making
• (WF2) Foster effective teamwork regardless of a team member’s
position of authority
5. Operations Focus
• Implement care delivery process improvements that avoid reliance
on memory and vigilance
6. Customer Involvement
• Engage patients and families in care delivery workflow process
design and feedback
7. Results
29
Type of Charts and Scoring
• All scores are ranked from 1.0 to 5.0; with 5.0
representing full implementation
• Domains & Key Aspects: Average Charts
• Average score for all respondents
• Cohort wide results
• Comparison results
• Individual Question Charts: %Positive Charts
• Scores 4-5: Positive
• Scores 1-3: Negative/Neutral
• Top 5 and bottom 5 rated items
Some Things to Keep in Mind….
The PSOA process is about continuous improvement
• Identify relative strengths and opportunities to improve
The conversations, sense making, and action are what matter
• Data is only useful if acted upon
The data you have is as good as the process you used to collect it
• The best data will include input from across levels of the
organization
The ‘Lake Woebegone Effect’: Where all the children are above average
• Self-ratings of performance are typically inflated, pay attention to
the range of scores and use data as a starting point for debriefing
Baseline data were collected through a shared sense-making approach
• Follow-up PSOA data represent an average across multiple
respondents from your organization
31
Bed size
4
Overall LADP Cohort 2
3
3
2
2
Bed size
Part of a larger system
Location
1
1
0
Fewer than 100 100 to 299 beds
beds
Part of a larger system
No
500 beds or
over
Location
3
3
Yes
3
3
0
1
2
3
Rural
Urban
3.78
Operations Focus
4.08
3.96
3.82
Workforce Focus
Customer Involvement
3.81
Teamwork
3.76
3.83
3.20
Reward for report
Measurement, Analysis,
Knowledge Mgmt
Pre six hospital average
Stratigic Planning
4.21
4.05
3.86
3.65
3.40
Best practice
Conduct assessment
Leadership
3.89
3.60
Promote non-punitive
culture
Safety as priority
Cohort 2 pre project result
4.40
4.20
4.00
3.80
Cohort 2 pre project result
5.00
Based on a 4
point scale
4.50
3.67
4.03
3.50
4.19
4.50
4.00
3.00
Reputation within
community
Stress patient safety Rate patient safety
Pre six hospital average
Recommend to
family member
Sample breakdown
No. of Participant of Pre PSOA
174
No. of Non LADP Participant
56
No. of LADP Participant
107
No. of Executive Leadership
51
No. of Unit Leadership
20
No. of Quality and Safety
43
4.04
3.93
Customer Involvement
3.99
3.75
Workforce Focus
3.98
3.70
Non LADP
Operations Focus
3.99
3.74
Teamwork
4.21
4.05
3.96
3.69
3.55
3.90
4.01
3.82
4.25
4.00
3.80
4.17
3.99
3.77
3.00
Reward for report
Measurement, Analysis,
Knowledge Mgmt
Stratigic Planning
Best practice
Conduct assessment
Leadership
Promote non-punitive
culture
4.32
4.20
3.50
Safety as priority
Respondents actively participating in LADP compared to
respondents not actively participating in LADP
4.50
4.00
LADP
Operations Focus
Workforce Focus
Teamwork
Reward for report
Measurement, Analysis,
Knowledge Mgmt
Stratigic Planning
3.54
4.01
3.87
3.97
3.75
3.51
3.98
3.92
3.93
4.16
4.07
3.96
3.93
4.04
3.82
3.53
3.56
3.35
3.84
3.80
4.07
4.06
3.96
4.06
3.97
3.82
3.83
3.71
3.50
3.20
3.00
Best practice
Conduct assessment
Leadership
Promote non-punitive culture
Safety as priority
3.50
4.10
4.30
4.24
Variation in PSOA Scores across leadership levels
4.50
4.00
Executive Leadership Average
Unit and Area Leadership Average
Frontline Average
Top 5 rated items
L14. One committee or senior leader oversees
patient safety within the organization.
77.14%
CF1. The organization informs and apologizes to
patients and their families when an adverse event
occurs.
81.18%
L15. Risk management, quality management, and
patient advocacy, are functionally integrated
around advancing patient safety.
84.91%
MAKM1. There is a safety alert communication
and dissemination system that gets information
to the right people in a timely fashion.
76.84%
L11. All departments, services, and standing
teams apply safety principles to work
deliverables.
86.79%
0.00%
% Positive
20.00%
40.00%
% Negative / Neutral
60.00%
80.00% 100.00%
Bottom 5 rated items
CF2. Patient information and education is
designed and delivered in useful formats,
matched to literacy and cultural needs.
52.38%
SP22. Lessons learned from healthcare and from
other industries are incorporated into the Patient
Safety Plan.
46.00%
SP15. The organization explicitly defines
employee and medical staff roles in advancing
patient safety in job descriptions, orientation,
and required continuing education.
50.00%
L25. Senior leadership directly communicates
with medical staff and employees using case
studies that illustrate a non-punitive approach to
adverse events.
52.88%
WF22. Following a patient safety adverse event
or near miss, the person involved is provided
non-punitive management support.
40.91%
0.00%
% Positive
20.00%
40.00%
% Negative / Neutral
60.00%
80.00% 100.00%
Section IV
Debriefing Your Patient Safety Organizational
Assessment (PSOA) Results
We have results…now what?
•Data is data
–Debriefing turns data into information
•Debriefing accelerates improvement
Units who used semi-structured
debriefing of culture survey:
10.2% Reduction in Infection
Rates
Units who did not
debrief survey results:
2.2% Reduction in
Infection Rates
Vigorito MC, McNicoll L, Adams L, Sexton B. Improving safety culture results in Rhode Island ICUs: lessons learned
from the development of action-oriented plans. Jt Comm J Qual Patient Saf. 2011 Nov;37(11):509-14.
41
PSOA Debriefing Tool:
What is it?
•
•
•
Structure for discussing specific actions to improve
safety
Provides questions that executive leaders and board
members can use to guide continuous improvement
Will help your team link the results of the PSOA
analysis to:
• improvement tools and materials we have
discussed throughout this project
The Johns Hopkins Armstrong Institute and VHA
Leadership Accountability Demonstration Project
The Chain of Accountability Model:
The Collaborative Tactical Decision Making Method
The Johns Hopkins Armstrong Institute and VHA
Leadership Accountability Demonstration Project
The Chain of Accountability Model:
Strategic Learning from Defects Tracer Tool
Step 1: Identify Key Aspects
of strength and
opportunities for
improvement. Record them
in the table below.
Overall Strengths:
List the 2 Key Aspects of
Safety with the highest
scores
Ex, Demonstrate patient
safety as a top leadership
priority (5.00)
Step 2: Drill down to identify
specific strengths and
specific opportunities for
improvement. Record them
in the table below.
Specific Strengths:
List the specific questions
within each “strength” rated
the highest
Ex.
1. Patient safety is adopted as
a strategic goal by the
organization and the
governing body. (5.00)
3. Risk management, quality
management, and patient
advocacy, are functionally
integrated around advancing
patient safety. (5.00)
7. All departments, services,
and standing teams apply
safety principles to work
deliverables. (5.00)
Overall Opportunities for
Improvement & Growth:
List the 3 Key Aspects of
Safety with the lowest
scores
Specific Opportunities for
Improvement & Growth:
List the specific questions
within each “opportunity to
improve” and the % of
respondents who responded
positively to each question
42
PSOA Debriefing Tool
(refer to the Complete Document)
• STEP 1: Your team identifies general strengths and opportunities for improvement.
• STEP 2: Your team identifies the specific behaviors and attitudes that make up those
strengths and weaknesses.
• STEP 3: Use questions provided to generate group discussion and discussion with board.
Your team chooses opportunities for growth, understanding that area of strength can
help address areas targeted for improvement.
• STEP 4: Your team identifies a strategy for addressing top opportunities selected in step
three.
• STEP 5: Your team works out the details of putting strategy into action.
• STEP 6: Your team evaluates plans. Be sure to place on monthly meeting agendas to
track progress, address barriers, and keep momentum moving
43
Next step: Action items
• HSOPS Debriefing
• Remember HSOPS assesses local culture
• Use the culture check up tool within the participating
unit, discuss it with frontline members that unit
• PSOA Debriefing
• PSOA assesses culture on organizational level and
identify pressing issues related to the project
• Use PSOA debriefing tool within the LADP project team
that includes members from all levels of the
organization
Discussion
Questions and Comments
Questions? Tools? Templates?
https://armstrongresearch.hopkinsmedicine.org/vha.aspx
Reminder…
You can access all slides,
call recordings, and project
tools at the VHA website