Writing up
improvement work for
impact and
implementation
Helen Crisp
Assistant Director
Research and Evaluation
Reporting for impact and implementation
Aim:
To encourage better
reporting of improvement
work to encourage
spread of successful
approaches
2
Reporting for impact and implementation
Why don’t we learn from
improvement projects?
Around the world 1,000s of improvement
projects in healthcare:
• Some make a difference
• Others don’t work
- but what do we learn from them?
3
Reporting for impact and implementation
We could learn, if . . .
Reports on improvement work told us not only
the results but also:
• how the initiative was designed
• what were the core components
• what measures and data were used to
measure the change
• what challenges were overcome along the
way
• how they were overcome
• what the team would do differently in the
future
4
Reporting for impact and implementation
Part of the problem
Many improvement reports amount to
“See we did X!”
versus
“Here’s what we had to do to achieve X”
Only your team (and possibly your parents) are
interested in the first
More people will care about the second
5
Reporting for impact and implementation
Writing up for implementation
Improvement reports need to provide enough
detail:
•
to convey credibly that something worked
•
to give others insight on what they would
have to do replicate in their clinical setting
6
Reporting for impact and implementation
Common problems
•
Reports lack important details about key
ingredients of intervention and institutional
context
− readers cannot know if it’s even worth pursuing
•
No barriers to implementation described
− No improvement effort works immediately, this
absence decreases credibility
− Readers will ask “How did you get clinicians to
do such and such?” Or “Did this newly
designed tool really work right away?”
7
Reporting for impact and implementation
Typical QI report
Introduction
Methods
Hospital infections affect
thousands each year
Hospital staff do not wash
their hands consistently
We implemented a multifaceted strategy:
• Staff education
Briefly stated design, data
collection strategy and
main outcomes, plus
some mention of PDSA
• Clinical champions
• Empowering patients to
ask staff if they have
washed their hands
Results
We improved hand
hygiene by 50%
Discussion
Patient empowerment can
be effective
8
Reporting for impact and implementation
What is lacking here?
Introduction
Hospital infections affect
thousands each year
Hospital staff do not wash
their hands consistently
We implemented a
multifaceted strategy:
• Staff education
No connection between
the introduction material
and
specific features of the
intervention
• Clinical champions
• Empowering patients to
ask staff if they have
washed their hands
9
Reporting for impact and implementation
A better approach?
Introduction
Commonly identified
barriers to hand hygiene
compliance include A, B,
and C
Staff education, clinical
champions, and
empowering patients
address A, B, and C by
doing X, Y, and Z
•
•
•
This introduction
makes clear what
factors explain poor
hand hygiene
And, it makes
explicit why the
intervention includes
these ingredients
This “theory for the
intervention” will pay
off in writing the
report and
interpreting the
results
10
Reporting for impact and implementation
But what did you do???
Methods
‘PDSA’ is not fairy dust!
Simply saying; ‘we
carried out three PDSA
cycles’ is not
informative. What did
the ‘study’ of what you
had ‘done’ reveal and
how did you ‘act’ as a
result?
Briefly stated design, data
collection strategy and
main outcomes, plus
some mention of PDSA
Results
We improved hand
hygiene by 50%
Discussion
Patient empowerment can
be effective
11
Reporting for impact and implementation
A better approach?
Method
After the first round of staff
education we reviewed the delivery
mechanism and feedback from
participants, using PDSA
methodology. It was reported that
timing of training sessions was an
issue in getting staff attendance,
so the next sessions were planned
with ward managers.
Participants wanted more visual
material to illustrate key points these were designed with staff and
used in subsequent sessions.
•
This provides more
detail which makes
the report credible
•
Others are likely to
have the same
issues and could
avoid making the
same mistakes
12
Reporting for impact and implementation
Points to consider
• When do you start ‘writing up?
• How to capture the key
ingredients of your
improvement initiative?
• How do you set appropriate
measures?
• When barriers arise
- how do you record these
- and the action to overcome
them?
13
Reporting for impact and implementation
Some tips for writing up
• Report writing always takes longer than you
think – guard against leaving it to the last
minute
• Robust data collection and reporting from the
outset is vital
• Keeping an ‘improvement diary’ of the work
helps to capture information at the time
• Include different perspectives - not just the
team leader, not just the clinical team
• Patients, service users, commissioners, other
departments/service providers will all have a
view
14
Reporting for impact and implementation
More tips for writing up
• Consider the audience
• While the core information is the same
presentation differs for:
• Accountability to the funder
• A record for the team and organisation
• Academic publication for researchers
• Practitioners looking to adapt your ideas for
local implementation
• Beyond text:
• Photos, videos, animation all bring the work to
life more than text ever will
15
Reporting for impact and implementation
A tool to help:
SQUIRE Guidelines
Standards for Quality Improvement Reporting Excellence
Checklist of points to consider when writing up
improvement work
BUT
Don’t leave it until you’ve completed the work
Use guidelines to consider what data to capture as you
go along
Developed in 2008 and currently being revised
16
Disseminating
and promoting
your work
17
Reporting for impact and implementation
Publication and beyond
• Using SQUIRE or other guidelines will help to
produce a coherent account
• This improves chances of external publication
• Approach a range of publications to target
different audiences
• Try to publish in the places professionals look
to get ideas (not just high-impact peer reviewed
journals)
• Other approaches:
•
Web-sites and resource centres
•
National awards and competitions
18
Reporting for impact and implementation
Awards
– what the judges look for:
• Clear results
• Tangible benefits for patients (and staff)
• Something a bit different – originality
• Understanding what was done and how
19
Reporting for impact and implementation
Spreading the word
Professionals listen to their peers!
Think of a range of approaches - and use every
opportunity:
• Blogs
• Twitter
• E-mail bulletins
and:
• Professional seminars
• National and international conferences
20
Challenge:
When you hear presentations on
improvement projects - if it’s not
clear how you would reproduce
the work, ask:
• More detail on the components?
• What were the challenges along
the way?
• What sort of training for staff?
• How did you engage the
patients?
• How did you use PDSA – what
did change as a result?
21
For more information on Health
Foundation funded improvement
programmes see:
www.health.org.uk
SQUIRE Guidelines can be
downloaded from:
http://squire-statement.org/guidelines
22