Audit and SEA Made Easy
GPST Teaching, Beardmore, November 2010
Mairi Jamieson
GPST Course Organiser
NHS Education for Scotland
Definitely not rocket science
Aims of workshop
 Refresh your knowledge of, or introduce you to,
audit and SEA
 Define criteria and standards
 Increase confidence in performing Audit/SEA in
your own practice
 Appreciate the use of audit/SEA in quality
assurance, also for revalidation, QOF etc.
 Signpost to resources for further guidance
What is audit?
What is Audit?
A range of definitions exist:
 Audit is about taking note of what we do, learning
from it and changing if necessary
 Audit is the improvement in the quality of care
through standard-setting, peer review,
implementation of change and re-evaluation
 Quite simply, audit is a tool that enables you to
monitor and then improve the quality of care you
provide to your patients.
 Systematic critical analysis of the quality of health
care
The Audit Cycle
Choose the Topic
Define Criteria & Standards
or
“What
What do
do you
you think
think you
you should
should be
be doing
doing”
Identify the Changes
Required &
Implement Them !!
Collect the Data
i.e.
The Information on
what you are doing
Assess Performance
against criteria & standards
How are we doing in relation to
what we should be doing ?
The Audit Cycle
Choosing The Topic
Eg. Clopidogrel Prescribing
The Audit Cycle
Define Criteria & Standards
Criteria - what you want to measure (yard-stick)
eg. Patients should have clopidogrel
prescribed only in accordance with NHSGGC
guidelines (ie. aspirin contraindicated or
postACS/angiography)
Standard - how well you should be doing
80% of patients should have clopidogrel
prescribed in accordance with NHSGGC
guidelines
The Audit Cycle
Collect the Data
1.
Identify patients on clopidogrel
2.
Pharmacist review of notes – identify
when started, by whom, indication,
whether ever on aspirin +/- PPI
3.
Determine whether in accordance
with guidelines
The Audit Cycle
Assess Performance
Compare our results with the
standard previously set
e.g. 21 of 116 patients on clopidogrel (18%)
were prescribed according to NHSGGC
guidelines – far below standard of 80%
The Audit Cycle
Agree & Implement
Changes Required
1.
Explore reasons for inappropriate
use
2.
Feedback to colleagues, discuss
changes and implement them
Eg. Letters to patients/cardiologists,
face-to-face medication review,
raising awareness of prescribers
The Audit Cycle
Repeat the Audit!!!
Data Collection 2
Repeat data collection once changes have had
a chance to take hold
The Audit Cycle
Re-assess Performance
Compare the results with the
standards previously set and results
of data collection 1
Has the standard now been met?
e.g. Now find that 48 of 90 (53%) of patients
on clopidogrel are prescribed within
guidelines
ie. Significant improvement but still below
standard
The Audit Cycle
Identify Further Changes Required
Long term issues:
Determine if further change is required to
sustain performance, and decide when
next to audit this topic (annually, every 2
years etc.)
Examples of Criteria & Standards
Criteria
Standard
Children under 2 years old should 90% of the registered 2 year olds
be immunized against tetanus
immunized against tetanus and
and polio
polio
The notes of those patients
sensitive to penicillin should
be clearly marked.
The notes of all (100%) patients
sensitive to penicillin clearly marked.
Patients should wait no longer
than 30 minutes in the surgery
before consultation.
75% of patients should wait no
longer than 30 minutes in the
surgery before consultation.
Criteria
Simple logical statements, used to describe a
measurable item of quality health care
ie. What you want to measure
e.g. Patients with asthma should have their
inhaler technique assessed at least once every
12 months.
Standard
Describes the ideal level of care to be
achieved for each criterion
ie. How well you should be doing
e.g. 80% of patients with asthma should have
their inhaler technique assessed at least once
every 12 months.
Arriving at Standards
 Don’t get overly concerned - standard setting is
flexible, can be revised upwards or down
 Those involved decide on the level of care they
find desirable - it is a professional issue.
 Guidance can be derived from the
literature/textbooks, but ultimately you decide.
 Can be based on your own work and
observations, varies between practices
Report format for audit
Report Section
Guidance
Reason for Audit
Explain why the topic was chosen
Potential benefits to patients/staff/practice
Criteria Set
Try to limit to 1-3, relevant to topic
Short simple logical statements
Justify with reference to current evidence
Standards Set
Agree a measureable standard for each
criterion
Set a realistic timescale
Preparation and
Planning
Who you discussed the audit with, and who
assisted you
How you collected and analysed the data
Report format cont.
Report Section
Guidance
Data Collection 1
Present using simple descriptive
statistics/tables/graphs
Do not present irrelevant data
Comment on comparison with standard set
Description of
Change
Describe changes agreed and implemented
Attach example as evidence if possible
Data Collection 2
Compare with standard and with data
collection 1
If standard not reached speculate as to why
Conclusions
What have you learned
How do you intend to take forward in future
Group Exercise 1
Criteria & Standards
Group Exercise 2
Implementing Change
Significant Event Analysis
What is a significant event?
“ Any event thought by anyone in the
team to be significant in the care of
patients or the conduct of the
practice “
(Pringle et al, 1995)
What happened?
 Record all of the facts relating to the identified
significant event (including any relevant dates,
times and people or organisations involved)
 Data source: those directly and indirectly involved
 Establish a clear and full picture of what happened
 Impact or potential impact
Why did it happen?
 Establish all of the main and underlying reasons
why the event actually occurred.
 Eg. A written telephone message about an
important meeting was not passed to the practice
manager because it had been lost.
But…. why was it lost?
Because it was written on a post-it and left on top of
a report, which was subsequently filed away by an
unsuspecting member of staff.
ie. Internal communication practices not up to
scratch!
What have you learned?
 Highlight any learning issues you and/or the
practice experience.
 For example it may be related to a training
need or a lack of knowledge concerned with
therapeutics, disease management or
administrative procedures.
 It could also reflect a learning experience (good
or not so good) in dealing with patients,
colleagues, or other organisations
 Ensure that insight into the event has been
established by the practice team or the
individuals concerned
What have you changed?
 Often a change in some aspect of care is required
to improve the provision of care and/or minimise
the risk that a similar event will occur.
 If so, a description of the change actually
implemented should be given rather than a “wish
list” of thoughts
What have you changed? (cont.)
 Sometimes it is not possible to implement
change, either because the likelihood of the
event happening again is so rare or because
change is outwith the control of the
individual/practice.
If this is the case, then reasons should be
clearly documented.
 Regardless of the type of significant event,
change should at least be considered, then
either implemented or justifiably ruled out
Important points
 Doesn’t have to be an bad event
 Could explore example of excellent practice
 Sharing
 Team activity
 Blame-free
 Constructive learning not finger-pointing
 Look beyond the superficial
 For underlying/systematic causes
“Nothing changes if nothing changes”
Group Exercise 3
Analysis of SEAs
Useful links for further information
 Guidance on Audit, RCGP Revalidation Toolkit (p28)
 http://www.rcgp.org.uk/PDF/Scot_Complete_Revalidation
_Toolkit_(Read_Only).pdf
 Ideas for Audit, NES
 www.clinicalgovernance.scot.nhs.uk/.../ideasforau
ditandSEA.rtf
 SEA – NPSA Guide 2008
 http://www.npsa.nhs.uk/nrls/improvingpatientsafety/prima
rycare/significant-event-audit/