Teaching for Quality
(Te4Q)
January 12, 2014
University of California, San Francisco
Introduction & Background:
why Te4Q?
Faculty
Dave Davis, MD
Senior Director, Continuing Education and Improvement, Association of
American Medical Colleges, Washington, DC
Jennifer S. Myers, MD FACP FHM
Associate Professor of Medicine
Director of Quality & Safety Education; Associate DIO for Quality, GME
Perelman School of Medicine, University of Pennsylvania
Philadelphia, PA
Participants & Projects: what’s
brought you here?
Getting the idea
Developing an educational innovation in Quality
Improvement/Patient Safety
Participant & Project
introductions..




Name
Faculty Role
Project Aim/Goal
Learners
the patient’s perspective
The clinical
care gap
The National view:
Dartmouth Atlas 2010 - HbA1c data
The TeQ Report:
Why Teaching
for Quality?
The Te4Q Vision:
Quality Improvement is
core to what it means to
be a physician
aamc.org/te4q
Te4Q Recommendation
“Every academic health
center will have a critical mass
of faculty ready, able and
willing to engage in, role
model, and teach about patient
safety and the improvement of
health care”
AAMC’s response:
The Te4Q Faculty Development Program
Moving ‘QI/PS savvy’ clinicians to expert QI educators
Clinical faculty as:
 Teachers
 Curriculum developers
 Evaluators, competency assessors
 Educational Quality Improvers
 Change Agents
 Leaders
 Role Models
 Mentors/peer advisers
The Te4Q Faculty Development Certificate Program
 Pre-Requisite: some knowledge of QI/PS
 Self- & Organizational-Readiness Assessments
 Pre-reading
 Skill Building Workshop

QI Educational Project w/presentation in 3 mo.

Community of Practice

Dissemination of Work—Presentation or
Publication

Certificate
Te4Q Workshop
Objectives
 Address an identified gap in the education of




students, residents, and/or practicing clinicians
regarding quality improvement and patient safety
Design an educational innovation to fill that gap
Effectively implement the initiative
Assess the impact of the innovation on learners and
the larger community
Enable and lead educational and organizational change
Workshop Agenda
Introductions
Adult Learning and Educational Principles
 Identify Gaps
 Learner Levels/Competencies
 Educational Program Goals and Objectives
 Interprofessional Education
 Educational Design: effective formats for learning
Reflection & Feedback
Developing QI/PS Content (JM)
 Frameworks for Teaching Quality
 What to Teach
 Teaching & Learning in the Clinical Environment
Workshop Agenda, cont’d
Assessing the Impact I
 Learner Assessment
 Formative vs Summative Feedback
Assessing the Impact II
 Program Evaluation
Reflection & Feedback
Making the Case & Leading Change
 Culture & the Hidden Curriculum
 Creating a Strategy for producing change
 Developing and implementation strategy
Dissemination and Next steps
Thank You
16
Section I:
Adult Learners:
setting goals, achieving competency in QI/PS
 Knowing Your Learner: Who are they?
 What are the core and supportive competencies in QI/PS?
 Assessing your own learning needs
 In time…developing effective educational goals and learning
objectives based on your learners needs
Subject Matter
Knowledge –
Clinical Medicine
Subject Matter Knowledge:
Knowledge basic to the things we do.
Professional knowledge: our clinical,
educational, and/or research
expertise
We Have a Lot of
These People
We Have Some of
SOI
Knowledge
These People
Knowledge of education:
Knowledge of adult learning
principles, curriculum development,
learner assessment, etc.
Knowledge of
Education
Subject Matter
Knowledge – Clinical
Medicine
We Have a Lot of
These People
Knowledge of
Education
We Don’t Have a Lot
of These
Subject
Matter People
Knowledge – Clinical
Medicine
Science of Improvement:
Knowledge of the theories of
systems, variation, measurement,
and psychology.
SOI Knowledge
Knowledge of
Education
Improvement occurs when we combine subject matter
knowledge and the science of improvement in creative ways to
develop effective ideas for change and then execute them
using proven quality improvement methods.
We Have a Precious
Few Number of
These People
Subject Matter
Knowledge
SOI Knowledge
Creating Quality & Safety Educators
Subject Matter
Knowledge – Clinical
Medicine
Knowledge of
Education
Quality & Safety
Educators
SOI Knowledge
We Do Not Have
Enough of These
People
What Type of Faculty Are Interested in
Teaching Quality & Safety?
* Quality and Safety Educators Academy – Attendee Roles
80
70
60
50
2012
40
2013
30
2014
20
10
0
Education Role
QI/PS Role
No Formal Role
Concepts of Competence:
what do we want our learners
to do? To ‘look like’?
Competency is defined as the observable behavior that
combines knowledge, skills, values, and attitudes related to a
specific activity
Meet Dr Zadowski
Emily Z is a 3rd year Internal
Medicine resident.
She meets you in Emerg to discuss Jim Halton, a 72 yr old
male with congestive heart failure. He needs re-admission,
has increasing symptoms of CHF (weight gain, increasing
SOBOE, orthopnea) - the result of inadequate care in his last
admission.
You and she review the notes from his admission and
discharge last week – including nursing and other
professional notes. You use the chart to stimulate her
observations about quality this case and others like it.
Is she competent in Quality? What
questions would you think to ask?
Common competencies:
the ACGME/ABMS Framework
 Medical Knowledge
 Patient Care
 Interpersonal Communication
 Professionalism
 Systems-based Practice
 Practice-based Learning and Improvement
In the Beginning…
The Mirror:
Practice Based Learning
& Improvement
The Village:
Systems-Based Practice
What Kind of Doctor Does our Health
System Need?
Old Model
New Model
Frank, et al. Lancet. 2010. 376: 1923-57.
Teamwork & Professionalism
Quality
Knowledge
Skills
Attitudes
IOM
Safe
Timely
Efficient
Effective
Pt centered
Equitable
QI methods,
Measurement,
& Tools
Stewardship
Value
Safety
Safety Principles
Error Types
Human Factors
Safety Tools:
RCA, FMEA
Safety Culture
Error Reporting
Leadership & Change Management
Health Information Technology
Evidence-Based Medicine
Myers, Tess, et al. Am J Med Qual. 2013
The checklist: Te4Q QI/PS “Proficient”
Competencies
 Critically evaluate and apply current healthcare information and
scientific evidence for patient care
 Systematically analyze practice using quality improvement
methods and demonstrate improvements in practice
 Working effectively in health care delivery settings, including
identifying system issues and improving them
 Incorporate considerations of cost awareness and risk-benefit
analysis in patient and/or population-based care
 Participate in identifying system errors and implementing
potential system solutions (patient safety)
 Work in interprofessional teams to enhance patient safety and
improve patient care quality
Dreyfus Model: Novice
Expert
Dreyfus and Dreyfus, 1982
Developmental Stages in QI/PS
Level
Training
Level
Novice
Pre-Clinical
medical student
Advanced
Beginner
Example
• Introductory lectures, web exercises
• Group work on case studies in QI/PS
• Students apply concepts (e.g., RCA) in a “project” at
Clerkship Student the academic health center
or Sub-Intern
•Teacher is model and “coach”
Competent
Residency &
Fellowship
Proficient
Fellowship &
Early practice
Expert
Advanced
practice/faculty
• Apply concepts to his or her own panel of patients in
interprofessional team
• Regularly review and improve care for patients
• Develop novel ways to understand and improve
systems of care
Two Types of Faculty Development
Type 1: Expert
Type 2: Proficient
Quality and
Safety
Educators
All Faculty*
Our Trainees
Need Both
Formal
Curriculum
Leading Experiential
Activities or
Immersion Activities
Informal
Curriculum
*All Faculty are clinical teachers and
role models for the residents
Faculty Learners: Skills We’d Like to Build
Proficient (Type 1)
Expert (Type 2)
Master
Core knowledge of QI/PS
Proficient, plus…
Expert, plus…
Common language
Increased experience in QI/PS
projects (eg. lead)
Curricular reform and/or clinical
leadership roles related to QI/PS
Leader in education and
curricular implementation
Scholarship in QI/PS
Doing basic improvement in practice
Modeling QI/PS principles with
learners
Prepared as a good improvement
team member
Participating in MOC Part IV
Other attributes…..
Career focus in QI/PS
Able to create experiential
and didactic learning
activities for students,
residents, others
Able to understand and
measure learner progress
Te4Q
A Look at Your Self Assessments
Knowing the Teacher
Self-Reflection Exercise:
• What is YOUR stage of development in QI/PS?
• What are your strengths
• Which skills need development?
Proficient
Expert
Master
Core knowledge of QI/PS
Proficient, plus…
Expert, plus…
Common language
Increased experience in QI/PS
projects (eg. lead)
Curricular reform and/or clinical
leadership roles related to QI/PS
Doing basic improvement in practice
Modeling w/learners
Prepared as good improvement team
member
Leader in education and curricular Scholarship in QI/PS
implementation
Career focus in QI/PS
Able to create experiential and
didactic learning activities for
students, residents, others
Participating in MOC Part IV
Able to understand and create
metrics to assess learner progress
Think About Your Project
Your Primary Learner
(Who Is Teaching? )
Knowl Subject
edge Matter
Type
Proficient
UME
-> Expert
Proficient
GME
-> Expert
Proficient
CPD
-> Expert
Science of
Improvement
Educational
Principles
Your Secondary Learner
(Who Is Learning?)
Subject
Matter
Novice
?
?
?
Science of
Improvement
Novice
?
?
?
Advanced Novice ->
Beginner -> Advanced
Competent Beginner
Proficient > Expert
?
Knowing your Learners
Team Think - Share
• Who are your faculty learners?
• What is their stage(s) of learning in the science of
improvement? in educational principles?
• Based on the above, what are their educational needs?
1. GOAL/AIM: What is the gap or problem you want to address through your
educational program? What are the Quality Improvement/Patient Safety competencies
you want your learners to exemplify at the completion of your educational program?
2. OBJECTIVES: What are the learning objectives? Use the term ‘performance
expectation’ to guide you here. What do you want learners to do after the activity?
Think SMART: specific, measurable, actionable, relevant and time bound
3. EDUCATIONAL DESIGN: What learning methods/formats will you use? Develop 2-3
bullet points to outline your educational idea, project or innovation and what QI/PS
content you want to include.
4. LEARNER ASSESSMENT: How will you assess your learners? What methods will
you use to assess what your learners have accomplished?
5. PROGRAM EVALUATION: How will you evaluate your initiative? How will you know
its impact and how it might be improved?
6. IMPLEMENTATION: How will you implement your project/innovation? Who will be
your partners? What resources will you need? From whom will you need ‘buyin”? How confident are you that you will be able to complete your project?
7. DISSEMINATION
Dissemination Plan: How will you report your project to a larger audience? eg. Poster,
presentation, publication.
BREAK
and evaluations
Adult learning Section II:
Creating effective
educational program and
innovations
the program planning cycle
 goals/objectives
 effective educational interventions
Comparing Educational Planning with Quality
Improvement
Educational Planning Cycle
1. Problem
Identification
& Needs
Assessment
5. Evaluation &
Feedback
4.
Implementation
2. Goals &
Objectives
3. Educational
Strategies
Kern: Curriculum Development for Medical Education 2009
Quality Cycle
Adapting problem and aim statement from
QI/PS: Developing Goals & Objectives
We’ve identified our problem and level of learner, and now….
Creating a Problem Statement
•Commonly used in both
Academic and Quality /
Performance Improvement
methodologies.
•Should meet the following
criteria:
•
•
•
•
Focused only on one problem
Only one or two sentences long
Should not suggest a solution
Unambiguous and devoid of
assumptions
| 43
|
43
Taking Aim
• Aim statements are very
specific declarations of what a
team will be focusing on as they
strive to improve a process or a
system.
• They should include a few
elements:
• The system to be improved
and the population
• A numerical goal
(preferably an ambitious
“stretch” goal)
• A timeframe
|
44
Your need/problem/goal
Goals/Objectives: Make them SMART
Creating Educational Objectives
1. First, a stem…
• After completing the seminar, residents will be able to….
• After this unit, the medical student will have . . .
• By completing the Year 3 curriculum, the health professional
student will…..
• At the conclusion of the course/unit/study the learner will . .
2. Next, add an action verb
• Use verbs from Bloom’s taxonomy list
• Determine the actual product, process, or outcome….
http://www.educationoasis.com/curriculum/LP/LP_resources/
lesson_objectives.htm
Interface of Learning Objectives and Teaching Strategies
case studies
projects
exercises
critiques
simulations
appraisals
Bloom’s
Taxonomy
lecture
visuals
video
audio
examples
illustrations
analogies
Cognitive
Processing
Dimension
Learning
Objectives
→
KNOWLEDGE
define
list
record
repeat
questions
discussion
review
test
assessment
reports
learner
presentations
writing
COMPREHENSION
describe
discuss
explain
express
Identify
recognize
restate
translate
exercises
practice
demonstrations
projects
sketches
simulations
role play
microteach
APPLICATION
apply
demonstrate
dramatize
employ
illustrate
interpret
operate
practice
perform
schedule
shop
sketch
use
problems
exercises
case studies
critical incidents
discussion
questions
test
ANALYSIS
analyze
calculate
compare
contrast
criticize
debate
diagram
differentiate
distinguish
experiment
inspect
inventory
question
relate
test
projects
problems
case studies
creative exercises
develop plans
EVALUATION
constructs
appraise
simulations
assess
choose
SYNTHESIS
estimate
arrange
evaluate
collect
judge
compose
measure
construct
rate
create
revise
design
score
formulate
select
organize
value
manage
plan
prepare
propose
set up
Adapted from: Bloom, B.S. (Ed.), Engelhart, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R. (1956). Taxonomy of educational objectives: The classification of
educational goals. Handbook 1: Cognitive domain. New York: David McKay.
Ellen F. Goldman, EdD 11-1-2010
Example of Program/Activity
Objective
Goal/Aim: After the full QI/PS course, 25% of all Internal Medicine and
Medicine-Pediatric residents will complete a longitudinal QI project with
general internal medicine faculty
•
AVOID
Vague verbs
“Know how to …”
•
General terms
CHOOSE
Explain, describe, discuss
Be as specific as possible
Examples of Learner Objective
Not so good:
At the end of this session, the resident will understand quality
improvement methodologies….
Suggestions for Improvement:
Better…
After completion of this course, learners in that
course will be able to:




define Root Cause Analysis* (RCA);
recognize which clinical situations require RCA;
explain why RCA is important;
demonstrate mutual respect on interprofessional
teams;
 and have performed an interprofessional mock RCA.
**a determination of the factors, processes and
problems leading to an error or care gap
Your Te4Q Educational Initiative
Team Think – Share:
•
Define/refine your learning objectives for your QI/PS
Initiative.
•
Share one of your objectives with us, your “consultants”
Interface of Learning Objectives and Teaching Strategies
Bloom’s
Taxonomy
lecture
visuals
video
audio
examples
illustrations
analogies
Cognitive
Processing
Dimension
Learning
Objectives
→
KNOWLEDGE
define
list
record
repeat
questions
discussion
review
test
assessment
reports
learner
presentations
writing
COMPREHENSION
describe
discuss
explain
express
Identify
recognize
restate
translate
exercises
practice
demonstrations
projects
sketches
simulations
role play
microteach
APPLICATION
apply
demonstrate
dramatize
employ
illustrate
interpret
operate
practice
perform
schedule
shop
sketch
use
problems
exercises
case studies
critical incidents
discussion
questions
test
ANALYSIS
analyze
calculate
compare
contrast
criticize
debate
diagram
differentiate
distinguish
experiment
inspect
inventory
question
relate
test
projects
problems
case studies
creative exercises
develop plans
constructs
simulations
SYNTHESIS
arrange
collect
compose
construct
create
design
formulate
organize
manage
plan
prepare
propose
set up
Adapted from: Bloom, B.S. (Ed.), Engelhart, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R. (1956). Taxonomy of educational objectives: The classification of
educational goals. Handbook 1: Cognitive domain. New York: David McKay.
Ellen F. Goldman, EdD 11-1-2010
case studies
projects
exercises
critiques
simulations
appraisals
EVALUATION
appraise
assess
choose
estimate
evaluate
judge
measure
rate
revise
score
select
value
Developing Educational Strategies &
Methods
Principles and Methods
What the theory of adult learning tells us..
Comfort/respect
Experience: (Kolb) build on
learner’s experience
Reflection: (Schon)
on past experience,
errors, prevention
Relevance: to current
Engagement: role play,
status, roles
simulations
What the research tells us
Physicians and others not self-aware: objective needs assessment,
performance feedback important
Knowledge necessary but not sufficient for change; didactics lousy
at changing performance
What works? Interactivity; sequencing; predisposing, enabling and
reinforcing strategies
‘CPD’ > conferences; = practice-based tools (reminders, audit-
feedback, protocols & training)
Docs pass through stages of learning: awareness, agreement,
adoption to adherence
…Cochrane reviews, AHRQ/EB reviews, others
Large Group: Interactive Lecturing
 Active participation: e.g.,
think-pair-share
 Lecturer=facilitator, docent,
group leader
 Widespread use of case, problems, vignettes
 NOITE: the Flipped classroom: reading and learning
before the session with application of knowledge at the
session.
Small Group Problem-Based Learning
 Generally 7-10 learners
 Uses case-based materials to stimulate discussion
• A patient safety issue, e.g., wrong site surgery and a
subsequent RCA or fishbone exercise
• A communication error in a team
• Other…your own
 Requires clear learning objectives, expectations of full
participation
 May use a tutor (expert or non-expert),
or be self-led
 Very useful for team development
Experiential Learning
Real-world experience
Simulations
- Handoffs
- Role play
- M&M conferences
- Standardized patients
- Rounds
- Simulation labs
- Bedside/Clinic
- Cases
- Computerized/games
Using an Tool Box
Quality Improvement Tools
The five
why’s
LEAN
thinking
Root cause
analysis
TeamSTEPS
Fishbone
diagrams
Process
mapping
Microsystem
planning
http://patientsafetyed.duhs.duke.edu/module_b/
module_overview.html
https://depts.washington.edu/toolbox/errors
Other
Sites
www.aamc.org/initiatives/cei/te4Q
Methods useful in Staging Learning – e.g., changing the
culture of quality in the workpace
Examples using the Pathman/PRECEED model
Stage/Method
Awareness
Predisposing
Lecture
Newsletter
Grand rounds
Meetings
Enabling
Reinforcing
Agreement
Adoption
Champions
Small group
discussion
Leadership buyin
Peer buy-in
Workshops
Training
sessions
Simulation
Mentorship,
coaching
Quality projects
Role-play
Feedback in
practice
Mentorship
Adherence
Reminders
Audit/Feedback
Celebration of
achievements
Teach Dr Zadowski
Emily Z is a 3rd year IM resident.
She meets you in Emerg to discuss Jim Halton, a 72 yr old
male with congestive heart failure. He needs re-admission.
You and she review the notes from his admission and
discharge last week – including nursing and other
professional notes. You use the chart to stimulate her
observations about QI/PS this case and others like it.
How would you prepare her and her classmates for
practices which focus on quality improvement and
patient safety? What educational programs, experiences
and other education can you create?
Core Entrustable Professional Activities for
Entering Residency (CEPARs)
•
EPA 13: Identify system
failures and contribute to a
culture of safety and
improvement
•
EPA 8: Give or receive a
patient handover to
transition care
responsibility
•
EPA 9: Collaborate as a
member of an
interprofessional team
1. GOAL/AIM: What is the gap or problem you want to address through your
educational program? What are the Quality Improvement/Patient Safety competencies
you want your learners to exemplify at the completion of your educational program?
2. OBJECTIVES: What are the learning objectives? Use the term ‘performance
expectation’ to guide you here. What do you want learners to do after the activity?
Think SMART: specific, measurable, actionable, relevant and time bound
3. EDUCATIONAL DESIGN: What learning methods/formats will you use? Develop 2-3
bullet points to outline your educational idea, project or innovation and what QI/PS
content you want to include.
4. LEARNER ASSESSMENT: How will you assess your learners? What methods will
you use to assess what your learners have accomplished?
5. PROGRAM EVALUATION: How will you evaluate your initiative? How will you know
its impact and how it might be improved?
6. IMPLEMENTATION: How will you implement your project/innovation? Who will be
your partners? What resources will you need? From whom will you need ‘buyin”? How confident are you that you will be able to complete your project?
7. DISSEMINATION
Dissemination Plan: How will you report your project to a larger audience? eg. Poster,
presentation, publication.
Section III:
Applying the Content of QI/PS to
educational program and
innovations
Teaching Quality Can Feel Like This
Who will teach ?
With what time ?
What to teach ?
How to teach ?
Where is the
Curriculum Guide?
Will the Residents “buy-in”?…
Will the Faculty “buy-in”? even “show up”?
Outline
How Can This Content be Organized?
• Educational Frameworks
What to Teach?
• Quality & Safety Content
How to Teach?
• Strategies for Effective Teaching of Quality & Safety
Our Reality: The “Middle Place”
Undergraduate & Graduate
Medical Education in
quality and safety is still
highly variable
Progress Has Been Slow
100
80
60
40
20
0
Patient Safety
QSEA 2012
Quality
Improvement
QSEA 2013
Inpatient
Handoffs
QSEA 2014
Discharge
Transitions
High Value
Cost
Conscious
Penn Fall 2013
Care
Common competencies:
the ACGME/ABMS Framework
 Medical Knowledge
 Patient Care
 Interpersonal Communication
 Professionalism
 Systems-based Practice
 Practice-based Learning and Improvement
Some Content Outlines
http://www.aacn.nche.edu/education-resources/ipecreport.pdf
http://qsen.org/
http://www.ihi.org/education/ihiopenschool/Pages/default.aspx
http://www.who.int/patientsafety/education/curriculum/EN_PSP_Education_Medical_Curriculum/en/
Quality & Safety in the ACGME Milestones
Good News:
…they mostly all say the same thing
Bad News:
…each specialty chose their own language to
describe their target competencies in:
•
•
•
•
Systems-Based Practice
Practice-Based Learning & Improvement
Interpersonal & Communication Skills
Professionalism
Example: Surgery
Systems-Based Practice Milestone
This resident participates in
groups or PI teams designed
to reduce errors & improve
health outcomes
This resident makes
suggestions for changes in
the healthcare system that
may improve patient care.
This resident reports
problems that could
produce medical errors.
SBP Milestones for a Graduating Resident
Medicine
Surgery
Emergency Med
Efficiently coordinates
activities of team members
to optimize care
Coordinates activities of
health care professional
team to provide optimal
care at time of discharge…
Coordinates system
resources to optimize
patient care for complicated
medical situations.
Identifies and advocates
for safe care and optimal
systems
Makes suggestions for
health system
improvement
Leads team reflections
such as de-briefings, RCAs,
or M&M to improve team
performance
Reports problems with
technology or processes
that could result in medical
error
Identifies situations when
breakdowns in teamwork
or communication could
lead to medical error
Participates in PI teams
designed to reduce errors
and/or improve care
outcomes
Understands appropriate
use of standardized
approaches to care &
contributes to them
Participates in an
institutional PI project to
optimize ED practice
Activates formal system
resources to investigate
or mitigate error
Advocates for costconscious utilization of
resources &
incorporates these
principles
Practices cost effective
care
The Milestones are mostly all at the
“Does” Level
Miller’s Pyramid for Learner Assessment
Outline
How Can This Content be Organized?
• Educational Frameworks
What to Teach?
• Quality & Safety Content
How to Teach?
• Strategies for Effective Teaching of Quality & Safety
Teamwork & Professionalism
Quality
Knowledge
Skills
IOM
Safe
Timely
Efficient
Effective
Pt centered
Equitable
Value
QI methods,
Measurement,
& Tools
Safety
Safety Principles
Error Types
Human Factors
Safety Tools:
RCA, FMEA
Safety Culture
Attitudes
Stewardship
Error Reporting
Leadership & Change Management
Health Information Technology
Evidence-Based Medicine
Myers, Tess, et al. Am J Med. 2013
Teach Dr Zadowski
Emily Z is a 3rd year IM resident.
She meets you in Emerg to discuss Jim Halton, a 72 yr old
male with congestive heart failure. He needs re-admission.
You and she review the notes from his admission and
discharge last week – including nursing and other
professional notes. You use the chart to stimulate her
observations about QI/PS this case and others like it.
How would you prepare her and her classmates for
practices which focus on quality improvement and
patient safety? What educational programs, experiences
and other education can you create?
Safety & Quality Are Connected
Afferent Arm
Safety event
Quality
Improvement
Opportunity
Efferent Arm
One Safety Event or Quality Problem:
Two Pathways For Teaching & Learning
Preventable Adverse Event,
Near Miss, Quality Problem
Patient
Safety
Methods
Retrospective Analysis
QI
Methods
Prospective Improvement
What Should We Review?
Both? Neither?
Patient Safety Methods
QI Methods
 Model for human error
 Hierarchy of Improvement Plans
 Cause-Effect Diagramming
 Event Reporting/Safety Culture
 Framing & scoping the
problem
 FOCUS-PDSA
 Process Mapping &
Observation
 Root Cause Tools
 Aim statements
 Measuring for Quality
Only Two Strategies to Eliminate Errors
• Find and fix systems
problems
• Plug up the “holes” in
the swiss cheese
• Mitigate and absorb
the natural human
error rate
• Promote safety
behaviors and culture
How Do Errors Happen?
Multiple Barriers - technology,
processes, and people - designed
to stop active errors (our “defense
in depth”)
BLUNT END
EVENT of
HARM
LATENT ERRORS
SHARP END
Are inherent properties
of the system that
allow or “set up” the
individual to fail
ACTIVE ERRORS are made
by individuals
Blunt vs Sharp
Active vs Latent
Reason, Managing the Risks of Organizational Accidents, 1997
Fishbone (Cause & Effect) Diagram
• Structured team
brainstorming
CAUSES
• Represents relationship
between some effect and
all of the possible causes
influencing it
People
Place
Cause
y
Wh
EFFECT
Why has room
turnover
time increased?
Policies
Five Whys
Processes
Environmental Services Taking Longer to Clean rooms
1st Why? New specialized room sterilization
2nd Why? High rates of resistant organisms
3rd Why? Some staff less familiar with the process
4th Why? No standardized training process for parttime, temporary staff
Hierarchy of Improvement Plans
How do you prevent customers from
leaving their ATM cards behind?
Strong Actions: Swipe card only
Intermediate Actions: Beeping
Weak Actions: Signs
Ranking the Effectiveness of
Error-Reduction Strategies
Most Effective (Strong)
–
–
–
–
–
–
–
Forcing functions and constraints
Automation and computerization
Standardization and protocols
Checklists and double-check systems
Rules and policies
Education and information
Exhortation: “Be more careful. Be vigilant.”
Least Effective (Weak)
Gosbee JW, Gosbee LL, eds. Human Factors Engineering to Improve Patient Safety.
Oakbrook IL: Joint Commission Resources 2005
One Safety Event or Quality Problem:
Two Pathways For Teaching & Learning
Preventable Adverse Event,
Near Miss, Quality Problem
Patient
Safety
Methods
Retrospective Analysis
QI
Methods
Prospective Improvement
How I Used to Do Quality Improvement
Different Quality Improvement Methodologies:
Similar Principles…All Foreign to Healthcare
A3 Diagram
FOCUS Before You Begin PDSA Cycles
D F
E
F
I
N
E
I
N
D
M O
E
A
S
U
R
E
R
G
A
N
I
Z
E
C
Problem Statement (from patient’s/customer’s viewpoint): (10 points)




Concise.
Customer- focused.
Addresses the business case. Shows why a change is needed.
Background Provides background for the problem statement (1-3 paragraphs with a minimum of 3
references using APA reference format. Why is this problem important? Who is interested in it (nationally
and/or locally) and why? Is there evidence behind this being a problem in healthcare? If so, provide a
summary of that evidence. [note: this item will be handed in typewritten, attached to your paper A3]
FOCUS
Find – Problem Statement
Organize – Current Condition
Clarify – Current Condition
Understand – Root Cause Tools
Current
Condition:-(20
points) Cause Tools…beginning to
Select
Root
 There is evidence that you did your “Go and See”.
select and think of countermeasures that
 The section is rich with data. Data is presented graphically – bar charts, run charts, or SPC charts are
preferred.
match the root cause(s)



You have at least one process map (Value-stream, flow chart, spaghetti diagram).
The metric(s) you are trying to move are clearly identified. Baseline/pre-test metrics are presented here.
Must consider at least one potential process, outcome, and balancing metric
You have completed a stakeholder analysis and attached it to your paper A3.
L
A
R
I
F
Y
A U
N
A
L
Y
Z
E
N
D
E
R
S
T
A
N
D
Target Condition: (10 points)



Plan – Plan your 1st Test of Change
Do – “Do” the Change
 Each countermeasure is clearly related to a root cause in the previous section.
Study
For each countermeasure,
how you
will DO the tests
- What, When and
by Whom.
– Track state
your
results
(metrics)
and
compare them to your
predictions
Propose and
Test
Act
–
Reflect
on
the test and decide what to
Countermeasures
do next…i.e. second test of
rd
Metrics/Results: (10 points) change, 3 , test of change,
eventually
implement
 If a test of change has been
implemented, the S.M.A.R.T.
goals are restated and results presented as data.



You use a relevant root cause tool. (5 Whys, Fishbone, Pareto). Must use at least one tool, though you may
need more than one for a deep analysis.
Your choice of which root cause(s) to tackle is supported by the data in the Current Condition section – i.e.
the data validates that you have identified the true root cause(s) and have chosen the right one(s) to work on
first.
PDSA
I
L
A
N
M
P
R
O
V
E
Propose and Test Countermeasures: (10 points)

Root Cause Analysis: (20 points)
Your target condition will move you towards delivering exactly what the customer wants, closer to IDEAL.
You identify at least one process, outcome, and balancing measure for your proposed project.
You have a clear aim statement, and your process and outcome metrics are expressed as S.M.A.R.T. goals.
P
Data should be in a run chart or a statistical process control chart (note: this is not required for successful
completion of the project for class; some students may not have this item completed yet; see next bullet)
If a test of change has not been implemented, describe which countermeasure you have chosen to test first
and why.
Reflect on What You Have Done and Learned so Far in Improvement work: (10 points)
Write a 1-3 paragraph reflection on what you learned by going through this process. What was most
challenging and why? What surprised you most and why? What do you think is your biggest challenge going
forward? [note: this will be handed in typewritten attached to your A3]
Make it Standard Work (Implement Successful Countermeasures):
While this step is not required for this class, be aware that failing to execute this step is extremely common in
healthcare. Skipping this step is usually the root cause for failing to sustain improvements after attention has moved
on to the next project. You should think about how you might implement a successful countermeasure and sustain any
improvements.
S
E
L
E
C
T
Executive Sponsor Initial Approval (signature and date):
Executive Sponsor Final Approval (signature and date):
Not required for this class – but never do a real project without this!
Not required for this class – but never do a real project without this!
A
N
D
D
O
C
C
H
E
C
K
O
N
T
R
O
L
or
S
T
U
D
Y
A
C
T
Outline
How Can This Content be Organized?
• Educational Frameworks
What to Teach?
• Quality & Safety Content
How to Teach?
• Strategies for Effective Teaching of Quality &
Safety
Building Bridges:
Linking Educational Activities With Health
System Improvement Efforts
Sound Familiar?
“Our hospital is struggling with the
over-utilization of labs and we have
had 2 RCAs on medication errors at
discharge. We just found out that the
GME office is starting QI education at
the program level and neither of these
topics was prioritized. Why doesn’t
anyone talk to each other around
here?”
“We cannot get a departmental
QI project off the ground
because we can’t get the
baseline data, let alone set up a
process for ongoing
measurement and evaluation.”.
-Associate Program Director
- Chief Quality & Safety Officer
“We have an incident reporting system, but I am pretty
sure the nurses are the only ones who use it. The
residents don’t know who reads them so are scared to
report. We also don’t see the faculty reporting so why
would we bother? We are busy enough.”
- Resident
Aligning Our Missions
Operational
Quality/Safety Goals:
Trainees as Front line
providers
Educational goals:
Trainees as learners
Residents as Common End Point
Barriers to Alignment and Capability are
Deeply Entrenched
Operational
Quality/Safety Goals:
Trainees as Front line
providers
Educational goals:
Trainees as learners
Residents as Common End Point
More Barriers
•Discontinuous trainee involvement
- Impedes the concept of ongoing system improvement
•Difficulties in scheduling and implementing essential
team-based interprofessional learning and improvement
‒ Simulation the default
•Failure to rush provide real-time data for improvement
to the front-line
•Lack of faculty trained in point-of-care experiential,
inter-professional learning techniques and improvement
science methods
•Tortuous and painful promotional pathway and reward
structure for such faculty
Slide adapted with permission from Don Goldmann MD, IHI
Will Require Shared Responsibility:
New Relationships, Roles, and Work
Daily supervision, role
modeling, & practice
enforces local
quality/safety culture
Frontline
Faculty & All
Staff
Curriculum
development,
Teaching,
Mentorship
Hospital
Quality
Office
Trainees
Infrastructure,
Shared Work Plan
GME
Office
Oversight
Centralized
resources
Core
QI/PS
Faculty
Tess, Vidyarthi, Yang, Myers [In Press]
“Top 5 List” for Healthcare
Organizations That Wish to Achieve
Excellence in Teaching for Quality
#1. Select Educational Content that
aligns with health care system needs
“ Top-Down” vs “Bottom-Up”
Approach to Project Selection
Penn Medicine
will improve the
health of our
patients and
assure safe care.
ENGAGEMENT
Involve faculty and
staff as partners
with patients and
families to achieve
goals of care.
CONTINUITY
Deliver seamlessly
coordinated care
across all settings
and service lines.
VALUE
Provide high quality,
efficient care and
the best outcomes
for all patients.
106
Resident Project Selection Aligns with
a Clinical Microsystem QI Need
Resident
Unit
QI Topic
Career Path
I.L.
Gen Med
Discharge Med Rec
Endocrine & CHIPS* fellow
M.P.
MICU
Post-Intubation Checklist
RWJ Scholar
I.W.
CCU
FMEA Door to Balloon Time
Cardiology Fellow
M.A.
L&D
Safety Culture and Team Debriefings
Urogyn Fellow
H.G.
Emerg Dept
Communication: Neurology & ED
Academic Emerg Med
S.G.
Medicine Clinic
Screening Colonoscopy Rates
Primary Care
Patel, Brennan, Myers, et al. Acad Med. 2014
*Center for Healthcare Improvement & Patient Safety, UPenn
#2: The Education Must Be “Hands-On”
• Real cases for quality improvement & patient safety learning
– Dissect a near miss or preventable adverse event using root cause
tools
– Use a current quality problem to:
•
•
•
•
Define and scope the problem
Write a specific aim statement
Identify process, outcome, and balancing measures
Identify root causes and select countermeasures
• Real data for QI review
– What type of data?
•
•
•
•
•
Healthcare associated infections
Patient satisfaction scores
Readmission rates
Test utilization
Other?
Data Matters
You Need to Consider:
How will I get these cases?
How/where will I get this data?
For this hands-on education, need to determine
the relative value of these goals -- to improve
care right now, to teach someone to improve
care, or both?
Examples of Each from Penn
• Improve care right now:
– On the general medicine teaching services, we have committed
to including the nurse in bedside rounds
• Teach someone to improve care
– 3rd year Penn Medical, Nursing, and Pharmacy students reflect
upon and dissect a preventable adverse event together using
root cause analysis tools
• Do Both
– 2nd year Penn internal medicine residents each review one of
their own 30-day readmissions looking for opportunities for
improvement to feed back to the Dept of Medicine Readmission
task force
– Role Modeling
#3. The Educators Must….
Make Connections for the Trainees
One success factor for trainee QI projects is faculty
mentors who have local system knowledge and deep
connections within the organization*
Ogrinc G, et al. Academic Medicine, 2014
What are we connecting the trainees to?
People in our
Organization
Data
Infrastructure
Residents & Fellows
Creating Quality & Safety Educators
Subject Matter
Knowledge – Clinical
Medicine
Knowledge of
Education
Quality & Safety
Educators
SOI Knowledge
Connections within their
health system
Assume trainees know nothing
(and I mean nothing)
about managing a project longitudinally
and running meetings
#4 Use a Common Language
• Make the “jargon” UCSF-branded jargon
• Takes awhile, but begins to catch on
• Helpful for teaching across the continuum and
sharing of educational materials
#5 Look for Every Opportunity to Make
Teaching & Learning Interprofessional
• Teaching about Health care delivery and
improvement is a natural IPE topics
• IP Quality & Safety Grand Rounds
• By definition, QI projects are interprofessional
Break: Apply the 5 Tips to Your Project
 Select Educational Content That Aligns With
Organizational Health Care Needs
“Hands-On” Education
Educators Making Connections For Trainees
Use a Common Language
Make it Interprofessional Whenever Possible
1. GOAL/AIM: What is the gap or problem you want to address through your
educational program? What are the Quality Improvement/Patient Safety competencies
you want your learners to exemplify at the completion of your educational program?
2. OBJECTIVES: What are the learning objectives? Use the term ‘performance
expectation’ to guide you here. What do you want learners to do after the activity?
Think SMART: specific, measurable, actionable, relevant and time bound
3. EDUCATIONAL DESIGN: What learning methods/formats will you use? Develop 2-3
bullet points to outline your educational idea, project or innovation and what QI/PS
content you want to include.
4. LEARNER ASSESSMENT: How will you assess your learners? What methods will
you use to assess what your learners have accomplished?
5. PROGRAM EVALUATION: How will you evaluate your initiative? How will you know
its impact and how it might be improved?
6. IMPLEMENTATION: How will you implement your project/innovation? Who will be
your partners? What resources will you need? From whom will you need ‘buyin”? How confident are you that you will be able to complete your project?
7. DISSEMINATION
Dissemination Plan: How will you report your project to a larger audience? eg. Poster,
presentation, publication.
#6: Remember that all formal curriculum
has a parallel “hidden” curriculum…
“QI”
…more on this tomorrow!