Components of a Learning System
David M. Williams, Ph.D.
Improvement Advisor
Institute for Healthcare Improvement
D
D
The Early Years Collaborative
5
• “Scotland will be the best place to grow up.”
• Draws upon existing community infrastructure across the country
• Focused on age-based workstreams:
• 15% reduction in the rates of still-births and infant mortality by
2015.
• 85% of all children within each CPP will reached all of the
expected developmental milestones at the time of the child’s
27-30 month child health review, by end-2016.
• 90% of all children within each CPP have reached all of the
expected developmental milestones at the time the child starts
primary school, by end-2017.
D
Components of a Learning System
1.
2.
3.
4.
5.
System level measures
Explicit theory or rationale for system changes
Segmentation of the population
Learn by testing changes sequentially
Use informative cases: “Act for the individual learn for
the population”
6. Learning during scale-up and spread with a
production plan to go to scale
7. Periodic review
8. People to manage and oversee the learning system
From Tom Nolan PhD, IHI
Workstream 1 Aim
To ensure that women experience positive
pregnancies which result in the birth of more
healthy babies as evidenced by a reduction of
15% in the rates of stillbirths
(from 4.9 per 1,000 births in 2010 to 4.3 per 1,000
births in 2015)
and infant mortality
(from 3.7 per 1,000 live births in 2010 to 3.1 per
1,000 live births in 2015).
Workstream 2 Aim
To ensure that 85% of all children within
each Community Planning Partnership have
reached all of the expected developmental
milestones at the time of the child’s 27‐30
month child health review, by end‐2016.
Workstream 3 Aim
To ensure that 90% of all children within
each Community Planning Partnership have
reached all of the expected developmental
milestones at the time the child starts primary
school, by end‐2017.
Big Aims
We are here!
Source: Brandon Bennett, IA
Overall Project Measures vs.
PDSA Cycle Measures
Data for Project Measures:
Achieving
Aim
Adapting
Changes
During
PDSA
Cycles
- Overall results related to the project aim (outcome,
process, and balancing measures) for the life of the project
Data for PDSA Measures:
- Just enough data
- Quantitative data on the impact of a particular change
- Qualitative data to help refine the change
- Subsets or stratification of project measures for
particular patients or providers
- Collect only during cycles
Run Chart - Data for Learning & Improvement
6.00
5.75
Pounds of Red Bag Waste
Measure
5.50
5.25
5.00
4.75
Median=4.610
4.50
4.25
4.00
3.75
3.50
3.25
1
2
3
Time
4
5
6
7
8
9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Point Number
PDSA
Measures
Guide
Learning
about our
testing.
Process
Measures
Guide Learning
about how our
testing is
improving
reliability of the
process.
Outcome
Measures
Guide
Learning
about how
the reliability
of the
process is
achieving our
aim.
Components of a Learning System
1.
2.
3.
4.
5.
System level measures
Explicit theory or rationale for system changes
Segmentation of the population
Learn by testing changes sequentially
Use informative cases: “Act for the individual learn for
the population”
6. Learning during scale-up and spread with a
production plan to go to scale
7. Periodic review
8. People to manage and oversee the learning system
From Tom Nolan PhD, IHI
Early Years Collaborative
R
P
Pre-work
A
P
D
Expert
meetings
A
D
A
D
S
S
S
LS1 2 day
Kickoff
P
2 day
LS
2 day
LS
2 day
LS
Etc…
Supports
Cluster
meetings
Key Changes
Improvement
Measures
Expert QI & Early Years faculty
Networking events
Listserv
Site Visits
Phone conf
Assessments
Monthly Reports via web
Oct 2012
Jan 2013
May 2013
Oct 2013
TBC
TBC
R
WORKSTREAM 1 (conception to 1 year)
Theory of what actions will
reduce infant mortality
Theory of what drives infant mortality
Aim
2⁰
1⁰
Poverty
Quality Of Home Environment
Domestic Abuse &
Violence
Societal Issues
Transport, Community
Capacity & Cultures
Reduce infant
mortality
Access To Services
Employment
Detailed aim:
To ensure that women
experience positive
pregnancies which
result in the birth of
more healthy babies as
evidenced by a
reduction of 15% in the
rates of:
•
•
Attachment
Post-birth actions
Improved money
management
Improved rate of
breastfed babies
Quicker diagnoses of
Neonatal Abstinence
Syndrome
Improved leadership,
culture and planning
Improved family
centred response
Improved stability /
permanence for LAC
Health
Improved identification
Parenting skills
Improved joint working
Smoking / Alcohol &
Drug Misuse
stillbirths (from 4.9
per 1000 births in
2010 to 4.3 per 1000
births in 2015)
infant mortality
(from 3.7 per 1000
live births in 2010 to
3.1 per 1000 live
births in 2015)
Workforce Issues
Improved teamwork,
communication, skills
and collaboration
Nutrition
Improved management
and quality of care
Improved sharing of
information
Improved access
Pre-birth actions
Mental health &
wellbeing
Identification &
reasons for current
resilience
Version: 06/03/2013
WORKSTREAM 2 (1 year to 30 months)
R
Theory of what actions will
ensure developmental
milestones are reached
Theory of what drives developmental milestones
Aim
2⁰
1⁰
Poverty
Societal Issues
Children have all
the developmental
skills and abilities
expected of a 2730 month old
Improved money
management
Domestic Abuse &
Violence
Improved child’s dental
health
Workforce Issues
Access To Services
Employment
Detailed Aim:
85% of all children
within each CPP
have reached all of
the expected
developmental
milestones at the
time of the child’s
27-30 month child
health review by
end-2016
Quality Of Home Environment
Transport, Community
Capacity & Cultures
Child’s physical &
mental health and
emotional
development
Health
Improving child nutrition
Improving brain
development and
physical play
Improved family centred
response
Improved stability /
permanence for LAC
Attachment
Early Learning & Play
Improved early
identification
Additional Support
Improved joint working
Level of education
Improved sharing of
information
Misuse of alcohol &
drugs
Carer’s physical &
mental health and
skills
Improved teamwork,
communication, skills
and collaboration
Nutrition
Improved management,
planning and quality of
services
Disabilities & Mental
health
Improved leadership,
culture and planning
Parenting skills &
knowledge
Identification & reasons
for current resilience
Version: 06/03/2013
R
WORKSTREAM 3 (30 months to start of primary school)
Theory of what actions will
ensure developmental
milestones are reached at
the start of primary school
Theory of what drives developmental milestones
Aim
2⁰
1⁰
Poverty
Quality Of Home Environment
Domestic Abuse &
Violence
Societal Issues
Transport, Community
Capacity & Cultures
Children have all
the developmental
skills and abilities
expected at the
start of primary
school
Detailed Aim:
90% of all children
within each CPP
have reached all of
the expected
developmental
milestones at the
time the child starts
primary school, by
end-2017
Workforce Issues
Access To Services
Employment
Early Learning & Play
Child’s physical &
mental health and
emotional
development
Improved uptake of
benefits
Improved child’s dental
health
Improving child nutrition
Improving brain
development and
physical play
Improved family centred
response
Health
Improved stability /
permanence for LAC
Attachment
Improved identification
Additional Support
Improved joint working
Level of education
Improved management,
planning and quality of
services
Misuse of alcohol &
drugs
Carer’s physical &
mental health and
skills
Improved teamwork,
communication and
collaboration
Nutrition
Improved sharing of
information
Disabilities & Mental
health
Improved leadership,
culture & planning`
Parenting skills &
knowledge
Identification & reasons
for current resilience
Version: 06/03/2013
WORKSTREAM 4 (Leadership)
R
Theory of what drives leadership support
Aim
1⁰
2⁰
Theory of what actions will
ensure leadership support
Build commitment with
partners to focus on delivery
Early Years
Collaborative is a
strategic priority &
underpins all
policy planning
and operational
activity
Provide the
Leadership System
to support quality
improvement
across the Early
Years
Collaborative
Early Years
Collaborative
values, culture and
behaviours are
modelled by all
leaders at all levels
Leaders illustrate how users
are included in design,
improvement, and delivery
of Early Years
Leaders facilitate change by
cultivating innovation from
intelligence, insights and
wisdom of people working
together
Leaders demonstrate their
ability to set direction and
engage and mobilise staff to
constantly improve quality
of service
Leaders can describe how
they personally maintain
early years focus within
their working environment
Early years executive and
operational leads are
identified
Detailed Aim:
Timely delivery of all
three workstream
“stretch aims”
CPPs communicate the EYC
with enthusiasm and
consistency
Infrastructure to
support delivery of
Early Years
Collaborative
Measurement plan and
priorities are established
and triangulation with other
key data
Spread plan is in place for
core and innovative work
Strategy for capturing,
celebrating and spreading
innovation
Establish an EYC
Implementation Committee
Ensure a feedback
mechanism for issues
raised in Walk-rounds
Ensure the development of a
measurement system used
to understand and drive
quality indicators
Assign a senior leader to
each improvement area
(Workstreams 1-3 and
measurement)
Establish Programme
Management and remove
barriers
Meet regularly with the
Implementation Committee
to track progress and
remove barriers
Display data that depicts
progress towards aim
Ensure that the senior team
participates in Walk-rounds
Place quality issues at the
top of senior leader meeting
agendas
Add Early Years
Collaborative and outcomes
to the CPP agenda
Version: 06/03/2013
Components of a Learning System
1.
2.
3.
4.
5.
System level measures
Explicit theory or rationale for system changes
Segmentation of the population
Learn by testing changes sequentially
Use informative cases: “Act for the individual learn for
the population”
6. Learning during scale-up and spread with a
production plan to go to scale
7. Periodic review
8. People to manage and oversee the learning system
From Tom Nolan PhD, IHI
An approach to achieving the IHI’s The Triple Aim for a given population - from
the perspective of a consumer health plan-less, FFS based, Medicare participating, not
for profit, hospital->health system…
A population, for which claims
data exists and achieving Triple
Aim results will not result in
perverse economic loss.
Cannot be defined by a clinical
condition (Diabetes) or issue
(readmissions).
Note: The size of the rectangles is meant to be indicative of
population size, not cost.
An approach to achieving the IHI’s The Triple Aim for a given population - from
the perspective of a consumer health plan-less, FFS based, Medicare participating, not
for profit, hospital->health system…
A population, for which claims
data exists and achieving Triple
Aim results will not result in
perverse economic loss.
A sub-population,
high cost and or
high utilization
people from the
larger population.
People who
have “fallen
through the
cracks” of our
“rescue-care”
system.
Cannot be defined by a clinical
condition (Diabetes) or issue
(readmissions).
Note: The size of the rectangles is meant to be indicative of
population size, not cost.
An approach to achieving the IHI’s The Triple Aim for a given population - from
the perspective of a consumer health plan-less, FFS based, Medicare participating, not
for profit, hospital->health system…
A population, for which claims
data exists and achieving Triple
Aim results will not result in
perverse economic loss.
A sub-population,
high cost and or
high utilization
people from the
larger population.
People who
have “fallen
through the
cracks” of our
“rescue-care”
system.
Cannot be defined by a clinical
condition (Diabetes) or issue
(readmissions).
Sub-groups, people
from the high cost
high utilization subpopulation that can
be stratified based
upon relatively
similar needs.
Sub-groups
based more on
needs and less
on conditions.
Note: The size of the rectangles is meant to be indicative of
population size, not cost.
An approach to achieving the IHI’s The Triple Aim for a given population - from
the perspective of a consumer health plan-less, FFS based, Medicare participating, not
for profit, hospital->health system…
A population, for which claims
data exists and achieving Triple
Aim results will not result in
perverse economic loss.
A sub-population,
high cost and or
high utilization
people from the
larger population.
People who
have “fallen
through the
cracks” of our
“rescue-care”
system.
Cannot be defined by a clinical
condition (Diabetes) or issue
(readmissions).
Sub-groups, people
from the high cost
high utilization subpopulation that can
be stratified based
upon relatively
similar needs.
Interventions intended to
address the needs of high
cost high utilization subgroups, Plan Do Study Act
cycles.
Sub-groups
based more on
needs and less
on conditions.
Some interventions will
work and some will not. All
should result in learning
and start on the smallest
practical scale.
Note: The size of the rectangles is meant to be indicative of
population size, not cost.
Components of a Learning System
1.
2.
3.
4.
5.
System level measures
Explicit theory or rationale for system changes
Segmentation of the population
Learn by testing changes sequentially
Use informative cases: “Act for the individual learn for
the population”
6. Learning during scale-up and spread with a
production plan to go to scale
7. Periodic review
8. People to manage and oversee the learning system
From Tom Nolan PhD, IHI
Repeated Use of the PDSA Cycle for Testing
Percent
100%
Changes That Result in
Improvement
Percent of Surgeries with Intraoperative Temp
Control
P
80%
D
90%
P
S
D
4
30%
S
2
D
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25
Obs
Daily % cases in control
P
P
Hunches
Theories
Ideas
A
S
D
A
Implementation of Change
P
A
P
D
S
D
Follow-up
Tests
Very Small
Scale Test
Sustaining the gains
S
Mini-measure tracks
improvement cycles
Spreading
A
1
0%
P
10%
A
3
20%
A
40%
S
D
5
50%
S
6
60%
A
70%
Wide-Scale Tests of Change
Sequential building of
knowledge under a wide range
of conditions
48
Scottish Borders - Run chart of run charts!
R
Scottish Borders - PDSAs completed
R
30
Test! Test! Test!
• Burt Sandeman’s Story Post LS1:
– The Challenge To Be Quick
• Burt Sandeman’s Story Post LS2:
– The F-Word
• Looked after two-year olds:
– My Prediction Was Wrong
Components of a Learning System
1.
2.
3.
4.
5.
System level measures
Explicit theory or rationale for system changes
Segmentation of the population
Learn by testing changes sequentially
Use informative cases: “Act for the individual learn for
the population”
6. Learning during scale-up and spread with a
production plan to go to scale
7. Periodic review
8. People to manage and oversee the learning system
From Tom Nolan PhD, IHI
Act with the Individual
• Dad’s Care
• Asset Based Community Development in N.
Ayrshire
Components of a Learning System
1.
2.
3.
4.
5.
System level measures
Explicit theory or rationale for system changes
Segmentation of the population
Learn by testing changes sequentially
Use informative cases: “Act for the individual learn for
the population”
6. Learning during scale-up and spread with a
production plan to go to scale
7. Periodic review
8. People to manage and oversee the learning system
From Tom Nolan PhD, IHI
Things to Consider to “Scale-up”
• Determine full scale at project setup and milestones to reach full
scale
• Different changes may require different scale-up strategies
• Consider different dimensions of structure
– Information technology
– Physical (e.g. space, equipment, capacity)
– Human resources (workforce organization and capabilities)
– Financial
– Learning system
• Use “5x” (5--25--125--625--3125---) thinking to predict/define the
structural issues and set a path forward for testing
– (What is working when testing with x that probably won’t work with 5x, ...?)
• Standardize processes (e.g. training, referral)
• Understand oversight requirements as the system grows
Components of a Learning System
1.
2.
3.
4.
5.
System level measures
Explicit theory or rationale for system changes
Segmentation of the population
Learn by testing changes sequentially
Use informative cases: “Act for the individual learn for
the population”
6. Learning during scale-up and spread with a
production plan to go to scale
7. Periodic review
8. People to manage and oversee the learning system
From Tom Nolan PhD, IHI
Period Review
Avg. Patient Satisfaction - Clinic A
Avg. Patient Satisfaction - Clinic B
Avg. Patient Satisfaction - Clinic C
70
60
50
40
30
20
80
70
Avg % Satisfaction
Avg % Satisfaction
Avg % Satisfaction
80
80
60
50
40
30
Q2
Q3
Q4
Q5
Q6
Q7
Q8
Q9
Q10
Q11
Q12
Q3
Q4
Q5
Q8
Q9
Q10
Q11
60
50
40
50
40
20
20
10
10
0
Q8
Q9
Q10
Q11
Q3
Q4
Q5
Q6
Q7
Q8
Q9
Q10
Q11
30
60
50
Q6
Q7
Last 12 Quarters
Q8
Q9
Q10
Q11
Q12
Q6
Q7
Q8
Q10
Q11
Q12
40
30
Q9
Q10
Q11
Q12
60
50
40
30
10
Q5
Q5
70
20
Q4
Q4
80
70
10
Q3
Q3
Avg. Patient Satisfaction - Clinic I
10
Q2
Q2
Last 12 Quarters
20
Q1
Q9
30
20
0
Q12
40
Q1
Avg % Satisfaction
Avg % Satisfaction
40
Q11
50
Q12
80
50
Q10
60
Avg. Patient Satisfaction - Clinic H
Avg. Patient Satisfaction - Clinic G
60
Q9
70
Last 12 Quarters
70
Q8
0
Q2
Last 12 Quarters
80
Q7
10
Q1
Q12
Q6
20
0
Q7
Q5
80
60
30
Q6
Q4
Avg. Patient Satisfaction - Clinic F
70
30
Q5
Q3
Last 12 Quarters
Avg % Satisfaction
70
Q4
Q2
Q12
80
Avg % Satisfaction
Avg % Satisfaction
Q7
Avg. Patient Satisfaction - Clinic E
80
Avg % Satisfaction
Q6
Last 12 Quarters
Avg. Patient Satisfaction - Clinic D
Q3
30
Q1
Q2
Last 12 Quarters
Q2
40
0
0
Q1
Q1
50
10
10
Q1
60
20
20
10
0
70
0
0
Q1
Q2
Q3
Q4
Q5
Q6
Q7
Last 12 Quarters
Q8
Q9
Q10
Q11
Q12
Q1
Q2
Q3
Q4
Q5
Q6
Q7
Last 12 Quarters
Q8
Keeping an eye on the journey
You are here!
The Work Remaining to Do
Aim
Components of a Learning System
1.
2.
3.
4.
5.
System level measures
Explicit theory or rationale for system changes
Segmentation of the population
Learn by testing changes sequentially
Use informative cases: “Act for the individual learn for
the population”
6. Learning during scale-up and spread with a
production plan to go to scale
7. Periodic review
8. People to manage and oversee the learning system
From Tom Nolan PhD, IHI