2011 Audit against the Key Performance Indicators
(KPIs) in the BASHH MedFASH STI Management
Standards (STIMS)
Updated presentation 20 Sept 2011
Hugo McClean
On behalf of the BASHH National
Audit Group
Slide 1 of 59
2011 STIMS Audit- presentation scheme
•
•
•
•
•
•
STIMS Key Performance Indicators (KPIs)
Methods
Results
Key messages
Areas for improvement
Use of STIMS Audit findings
Slide 2 of 59
2011 Audit against the BASHH MedFASH
STI Management Standards (STIMS)
• Standards:
• Standards for the management
of sexually transmitted
infections
• Posted on BASHH and MedFASH
websites Jan 2010
• Aims: “to support the
commissioning and provision of
high quality care for STIs across
all settings”
Slide 3 of 59
STIMS KPIs
• Practice in 9 patient management and policy areas
• Many required further definition to allow auditing
• (Continuing role of BASHH Clinical Standards Unit,
Dr Immy Ahmed)
Slide 4 of 59
STIMS KPIs
Patient management: data from case notes
KPI
Summary
Further definition
needed
1.4.1
3.4.1
3.4.1
48 hour access (patient registers)
Sexual history
STI/HIV risk assessment
No
Yes
Yes
3.4.2
4.4.1
5.4.1
HIV testing & uptake
Test results within 7 working days
Partner notification
No
Yes
Yes
Slide 5 of 59
STIMS KPIs
Clinic policies
KPI
Summary
Further definition
needed
2.4.1
6.4.1
7.4.1
Staff competency/updating
Information governance
Care pathways to Level 3 services
Yes
Yes
Yes
8.4.1
9.4.1
Audit participation and planning
Patient and Public Engagement plan
No
Yes
Slide 6 of 59
Methods- eligible services
• Eligible services:
– All UK Nations
– Genitourinary medicine clinics
– Services providing STI management at either level 2 or 3 defined in the
BASHH MedFASH STIMS Project definitions in STIMS Appendix B
• Faculty of Sexual & Reproductive Healthcare consultant-led services
• Primary Care practitioner-led services at level 2
• Commissioned by PCTs to be provided by the independent or third sectors
• Not included:
– Pharmacy-based services
– National Chlamydial Screening Programme services
– Non PCT-commissioned independent or third sector services
Slide 7 of 59
Methods- principles
• Clinics seeing smaller numbers of cases
• Senior clinical staff asked to assist with data
collection
• Asked to seek help from managers for
information governance questions
• Informed of planned re-audit in 2014
• Case note data returned at clinic level
Slide 8 of 59
Methods: data collection- 1
• Audit interval:
– Clinic policies: as of 31 December 2010
– Cases seen 1 October to 31 December 2010
• Data collection
– Launched 6 Jan 2011
– Closed 30 April 2011
– Complete data set presented
• Participation:
– Level 2 services: PCT Sexual Health Leads via Andrea Duncan
(DH Sexual Health & HIV Programme Manager)
– Level 3 services: BASHH NAG Regional Chairs network
– BASHH website
• Comments in free text boxes for each question: BASHH website
Slide 9 of 59
Data collection- 2
• Data collected by clinics in an Excel workbook
• Data submitted using an online form
Slide 10 of 59
Data collection- 3
• Clinic policy data
• Case note data:
– Up to consecutive 40 cases*
– Clinic level performance
computed: no individual
patient data
• Free text comments
*RCP Local Clinical Audit: handbook for
physicians: http://old.rcplondon.ac.uk/clinicalstandards/ceeu/Documents/Local-clinical-audithandbook-for-physicians-August-2010.pdf
Slide 11 of 59
Data collection- 4
• Data transcribed and
submitted using an online
form:
Slide 12 of 59
Results- clinic level data
• Clinic level data
• Percentage of clinics in each BASHH Region
meeting KPI performance target
• Overall national performance
Slide 13 of 59
Number (%) of clinics participating, by Region
BASHH Audit Group Region
North Thames
South Thames
Trent
South West
Yorkshire
Anglia
Wales
Northern
West Midlands
Oxford
Northern Ireland
Wessex
Cheshire and Merseyside
North West
National (%)
Slide 14 of 59
Level 3 (%)
27 (18%)
24 (16%)
15 (10%)
12 (8%)
12 (8%)
10 (7%)
10 (7%)
8 (5%)
8 (5%)
6 (4%)
5 (3%)
5 (3%)
4 (3%)
3 (2%)
149 (100%)
Level 2 (%)
4 (11%)
10 (27%)
3 (8%)
3 (8%)
6 (16%)
6 (16%)
3 (8%)
None
None
1 (3%)
None
None
None
1 (3%)
37 (100%)
Total (%)
31 (17%)
34 (18%)
18 (10%)
15 (8%)
18 (10%)
16 (9%)
13 (7%)
8 (4%)
8 (4%)
7 (4%)
5 (3%)
5 (3%)
4 (2%)
4 (2%)
186 (100%)
KPI 1. 48 hour access
• Percentage of people offered an appointment,
or seen by a healthcare worker on walking-in,
within 48 hours of contacting an STI provider
• Standard 98%
Slide 15 of 59
KPI 1. Percentage of clinics in each Region with ≥98%
48 hour access
National L3=149, L2=37
West Midlands L3=8, L2=0
Wessex L3=5, L2=0
Northern L3=8, L2=0
North West L3=3, L2=1
Cheshire & M'side L3=4, L2=0
Anglia L3=10, L2=6
North Thames L3=27, L2=4
South Thames L3=24, L2=10
Trent L3=15, L2=3
South West L3=12, L2=3
Wales L3=10, L2=3
Yorkshire L3=12, L2=6
Oxford L3=6, L2=1
N. Ireland L3=5, L2=0
Level 3, n=149
Level 2, n=37
Slide 16 of 59
86%
51%
100%
100%
100%
100%
100%
100%
100%
50%
96%
100%
88%
50%
87%
33%
83%
70%
100%
67%
17%
67%
100%
20%
0%
20%
40%
60%
80%
Percentage of clinics meeting Standard
100%
KPI 2. Staff competency
• Percentage of staff who have completed
competency-based training
• No nationally agreed standards for competency
for some clinical staff groups
• Percentage of staff with documentation of
competency
• Question: no justification of how competency
was achieved, only whether it was documented
• Standard 100%
Slide 17 of 59
KPI 2. Percentage of clinics in each Region with 100% staff
with competency documented
National L3=149, L2=37
South Thames L3=24, L2=10
Yorkshire L3=12, L2=6
Cheshire & M'side L3=4, L2=0
North Thames L3=27, L2=4
Wessex L3=5, L2=0
Anglia L3=10, L2=6
West Midlands L3=8, L2=0
Northern L3=8, L2=0
Oxford L3=6, L2=1
North West L3=3, L2=1
South West L3=12, L2=3
Wales L3=10, L2=3
Trent L3=15, L2=3
N. Ireland L3=5, L2=0
Level 3, n=149
Level 2, n=37
Slide 18 of 59
40%
62%
58%
50%
67%
50%
40%
40%
38%
100%
48%
50%
50%
38%
33%
33%
0%
0%
100%
25%
20%
33%
20%
67%
20%
0%
20%
40%
60%
80%
Percentage of clinics meeting Standard
100%
KPI 2. Staff updating
• Percentage of staff who have fulfilled update
requirements
• No nationally agreed standards for update
requirements
• Percentage of staff with documentation of updating
• Question: no justification of how updating was
achieved, only whether it was documented
• Standard 100%
Slide 19 of 59
KPI 2. Percentage of clinics in each Region with 100% staff
with updating documented
54%
National L3=149, L2=37
Yorkshire L3=12, L2=6
South Thames L3=24, L2=10
North Thames L3=27, L2=4
West Midlands L3=8, L2=0
Wessex L3=5, L2=0
Oxford L3=6, L2=1
Northern L3=8, L2=0
Anglia L3=10, L2=6
Trent L3=15, L2=3
South West L3=12, L2=3
North West L3=3, L2=1
Wales L3=10, L2=3
Cheshire & M'side L3=4, L2=0
N. Ireland L3=5, L2=0
Level 3, n=149
Level 2, n=37
Slide 20 of 59
65%
67%
50%
67%
90%
67%
100%
63%
60%
50%
100%
50%
33%
50%
50%
47%
42%
100%
33%
0%
30%
0%
25%
20%
0
0.2
0.4
0.6
0.8
Percentage of clinics meeting Standard
1
KPI 3. Sexual history
• KPI: “Percentage of individuals with STI concerns who had a sexual history taken.
Standard 100%”
• Case definition: eligible for STI screening because of concern about STIs
spontaneously expressed, or elicited during the visit to a service (either verbally
or indicated on a triage or similar form)
• Scoring system based on data elements from:
– BASHH 2006 National guidelines on undertaking consultations requiring sexual history
taking*
• Competency (if age <16 years)
• Symptoms
• Reason for attendance
• Sexual contact details
• Previous STIs
• First day of last menses, or documentation about vaginal bleeding (women)
• Contraception (women)
• Cervical cytology (women age >=25 years)
– Score weighting based on gender and age
• Standard: 75% documentation based on questions
Slide 21 of 59
KPI 3. Percentage of clinics in each Region with
≥75% sexual history documentation
National L3=149, L2=37
North West L3=3, L2=1
Cheshire & M'side L3=4, L2=0
North Thames L3=27, L2=4
South West L3=12, L2=3
South Thames L3=24, L2=10
West Midlands L3=8, L2=0
Northern L3=8, L2=0
Wales L3=10, L2=3
Anglia L3=10, L2=6
Yorkshire L3=12, L2=6
Trent L3=15, L2=3
Wessex L3=5, L2=0
N. Ireland L3=5, L2=0
Oxford L3=6, L2=1
Level 3, n=149
Level 2, n=37
Slide 22 of 59
77%
59%
100%
100%
75%
33%
93%
83%
79%
70%
75%
75%
70%
100%
70%
33%
67%
67%
67%
33%
60%
60%
50%
0%
20%
40%
100%
60%
80%
Percentage of clinics meeting Standard
100%
KPI 3. STI/HIV risk assessment
• KPI: Percentage of individuals with STI concerns who had a STI/HIV risk
assessment made. Standard 100%
• Case definition: eligible for STI screening because of concern about STIs
spontaneously expressed, or elicited during the visit to a service
• Scoring system based on data elements from:
– BASHH 2006 National guidelines on undertaking consultations requiring
sexual history taking*
•
•
•
•
•
•
Lifetime injecting drug use
Sex abroad
Risk factors for hepatitis B
Medical treatment abroad
HIV testing history
Lifetime sexual contact with another man (men)
– Score weighting based on gender
• Standard: 75% documentation based on questions
Slide 23 of 59
KPI 3. Percentage of clinics in each Region with ≥75%
STI/HIV risk assessment documentation
National L3=149, L2=37
Northern L3=8, L2=0
South West L3=12, L2=3
Wessex L3=5, L2=0
West Midlands L3=8, L2=0
South Thames L3=24, L2=10
N. Ireland L3=5, L2=0
Wales L3=10, L2=3
North Thames L3=27, L2=4
Yorkshire L3=12, L2=6
Oxford L3=6, L2=1
North West L3=3, L2=1
Anglia L3=10, L2=6
Cheshire & M'side L3=4, L2=0
Trent L3=15, L2=3
Level 3, n=149
Level 2, n=37
Slide 24 of 59
56%
43%
88%
83%
33%
80%
75%
63%
50%
60%
60%
100%
59%
0%
58%
33%
50%
100%
33%
0%
30%
50%
25%
13%
0%
20%
33%
40%
60%
80%
Percentage of clinics meeting Standard
100%
KPI 3. HIV testing & uptake
• Case definition: eligible for STI screening because of
concern about STIs spontaneously expressed, or elicited
during the visit to a service
• Standards:
–Offer 100%
–Uptake, by those offered, 60%
Slide 25 of 59
KPI 3. Percentage of clinics in each Region with
100% HIV test offer
National L3=149, L2=37
Wessex L3=5, L2=0
Anglia L3=10, L2=6
West Midlands L3=8, L2=0
Northern L3=8, L2=0
North Thames L3=27, L2=4
Trent L3=15, L2=3
Wales L3=10, L2=3
South West L3=12, L2=3
South Thames L3=24, L2=10
Yorkshire L3=12, L2=6
Oxford L3=6, L2=1
North West L3=3, L2=1
Cheshire & M'side L3=4, L2=0
N. Ireland L3=5, L2=0
Level 3, n=149
Level 2, n=37
Slide 26 of 59
43%
50%
80%
70%
67%
63%
63%
56%
25%
53%
50%
33%
30%
33%
33%
33%
0%
67%
67%
50%
46%
33%
100%
25%
20%
0
0.2
0.4
0.6
0.8
Percentage of clinics meeting Standard
1
KPI 3. Percentage of clinics in each Region with ≥60% HIV
test uptake (of those offered)
National L3=149, L2=37
West Midlands L3=8, L2=0
Wessex L3=5, L2=0
Oxford L3=6, L2=1
N. Ireland L3=5, L2=0
North West L3=3, L2=1
North Thames L3=27, L2=4
Anglia L3=10, L2=6
South Thames L3=24, L2=10
Yorkshire L3=12, L2=6
South West L3=12, L2=3
Northern L3=8, L2=0
Trent L3=15, L2=3
Wales L3=10, L2=3
Cheshire & M'side L3=4, L2=0
Level 3, n=149
Level 2, n=37
Slide 27 of 59
94%
70%
100%
100%
100%
100%
100%
100%
100%
100%
50%
100%
67%
96%
50%
92%
100%
92%
67%
88%
87%
67%
80%
100%
75%
0%
20%
40%
60%
80%
Percentage of clinics meeting Standard
100%
KPI 4. Test results within 7 days
• Standard
– “Percentage of reports (or preliminary reports) that
are received by clinicians within 7 working days of a
specimen being taken”
• Further definition:
– Chlamydial test results chosen
– 'Received' = date report accessible to a relevant
clinician, either paper report, or electronically.
– Paper reports- date stamped on the report
– Electronic reports- date report electronically posted
by laboratory
Slide 28 of 59
KPI 4. Percentage of clinics in each Region with 100%
positive chlamydia test results within 7 days
National L3=149, L2=37
Wessex L3=5, L2=0
West Midlands L3=8, L2=0
Northern L3=8, L2=0
South Thames L3=24, L2=10
South West L3=12, L2=3
Anglia L3=10, L2=6
North Thames L3=27, L2=4
Oxford L3=6, L2=1
North West L3=3, L2=1
Trent L3=15, L2=3
Cheshire & M'side L3=4, L2=0
Wales L3=10, L2=3
N. Ireland L3=5, L2=0
Yorkshire L3=12, L2=6
Level 3, n=149
Level 2, n=37
Slide 29 of 59
39%
57%
80%
75%
63%
46%
33%
70%
42%
40%
33%
37%
75%
33%
100%
33%
0%
27%
0%
25%
20%
100%
20%
17%
0%
20%
67%
40%
60%
80%
Percentage of clinics meeting Standard
100%
KPI 5. Partner notification
• Standard
– “Rate of partner notification for chlamydia and gonorrhoea for each STI provider”
– Standard:
– At least 0.4 contacts per index cases in large conurbations, 0.6 elsewhere
– Within four weeks
• Measured for chlamydial infection
• Further definition:
– Contact event = seen for management
– Resolution: both verified by a healthcare worker AND reported by an index case
– Verified = contacting another agency if necessary
– Four weeks start = from date of first PN interview
• See slide 52 for London and outside-London PN performance
Slide 30 of 59
KPI 5. Percentage of clinics in each Region with ≥0.4
chlamydial contacts verified by a healthcare worker
National L3=149, L2=37
Wessex L3=5, L2=0
North West L3=3, L2=1
Wales L3=10, L2=3
Trent L3=15, L2=3
Northern L3=8, L2=0
Cheshire & M'side L3=4, L2=0
Yorkshire L3=12, L2=6
Oxford L3=6, L2=1
West Midlands L3=8, L2=0
N. Ireland L3=5, L2=0
Anglia L3=10, L2=6
South Thames L3=24, L2=10
North Thames L3=27, L2=4
South West L3=12, L2=3
Level 3, n=149
Level 2, n=37
100%
90%
67%
0%
100%
100%
87%
75%
75%
67%
50%
67%
0%
63%
60%
50%
50%
30%
0%
0%
60%
44%
75%
42%
20%
Null returns: Level 2, 49%; Level 3, 7%
Slide 31 of 59
63%
41%
40%
60%
80%
Percentage of clinics meeting Standard
100%
KPI 5. Percentage of clinics in each Region with ≥0.6
chlamydial contacts verified by healthcare workers
National L3=149, L2=37
Trent L3=15, L2=3
North West L3=3, L2=1
Wessex L3=5, L2=0
Wales L3=10, L2=3
Northern L3=8, L2=0
Anglia L3=10, L2=6
Yorkshire L3=12, L2=6
N. Ireland L3=5, L2=0
Oxford L3=6, L2=1
West Midlands L3=8, L2=0
South Thames L3=24, L2=10
Cheshire & M'side L3=4, L2=0
North Thames L3=27, L2=4
South West L3=12, L2=3
Level 3, n=149
Level 2, n=37
38%
35%
67%
100%
60%
50%
33%
50%
42%
40%
50%
50%
50%
33%
0%
25%
25%
20%
25%
22%
0%
0%
75%
17%
20%
Null returns: Level 2, 49%; Level 3, 7%
Slide 32 of 59
73%
0%
40%
60%
80%
Percentage of clinics meeting Standard
100%
KPI 5. Percentage of clinics in each Region with ≥0.4
chlamydial contacts reported by index cases
National L3=149, L2=37
South Thames L3=24, L2=10
Oxford L3=6, L2=1
North Thames L3=27, L2=4
Wessex L3=5, L2=0
Yorkshire L3=12, L2=6
Anglia L3=10, L2=6
North West L3=3, L2=1
West Midlands L3=8, L2=0
N. Ireland L3=5, L2=0
Wales L3=10, L2=3
Trent L3=15, L2=3
South West L3=12, L2=3
Northern L3=8, L2=0
Cheshire & M'side L3=4, L2=0
Level 3, n=149
Level 2, n=37
88%
40%
83%
81%
50%
100%
80%
75%
33%
70%
50%
67%
0%
63%
60%
60%
33%
33%
60%
67%
50%
50%
25%
0
0.2
Null returns: Level 2, 41%; Level 3, 5%
Slide 33 of 59
70%
43%
0.4
0.6
0.8
Percentage of clinics meeting Standard
1
KPI 5. Percentage of clinics in each Region with ≥0.6
chlamydial contacts reported by index cases
National L3=149, L2=37
Wessex L3=5, L2=0
South Thames L3=24, L2=10
North Thames L3=27, L2=4
Wales L3=10, L2=3
Anglia L3=10, L2=6
Trent L3=15, L2=3
Yorkshire L3=12, L2=6
N. Ireland L3=5, L2=0
West Midlands L3=8, L2=0
Oxford L3=6, L2=1
South West L3=12, L2=3
North West L3=3, L2=1
Northern L3=8, L2=0
Cheshire & M'side L3=4, L2=0
Level 3, n=149
Level 2, n=37
80%
0%
75%
40%
50%
33%
33%
63%
60%
60%
33%
67%
47%
42%
40%
38%
33%
100%
33%
33%
33%
0%
25%
0%
0%
0%
20%
Null returns: Level 2, 41%; Level 3, 5%
Slide 34 of 59
52%
41%
40%
60%
80%
Percentage of clinics meeting Standard
100%
National PN performance: percentage of clinics with
≥0.4 & ≥0.6 performance levels
Level 3,
n=164
Verified
Reported
Performance level: contacts
seen/index cases
≥0.4
≥0.6
% Clinics with this level of
performance:
63%
38%
70%
52%
Level 2,
n=37
Verified
Reported
41%
43%
Service
level
Slide 35 of 59
Verified/reported
35%
41%
Level 3 PN performance: 2011 vs 2007
Audit
2011 Standards
Audit
Performance level: contacts/index cases
≥0.4
≥0.6
Verified
Reported
Verified
Reported
% Clinics with this level of performance:
63%
70%
38%
52%
55%
52%
31%
25%
(contacts seen)
2007 BASHH
Chlamydia Audit
(contacts screened)
Slide 36 of 59
KPI 6. Information governance
• BASHH Standards: “Provision of data by all providers
of services managing STIs complies with national and
local reporting requirements”
• Please check with your service manager and/or
senior clinical staff to help answer these questions!
• Scoring system based on 22 information governance
components
• Standard 100%
Slide 37 of 59
KPI 6. Information governance- questions
Slide 38 of 59
KPI 6. Percentage of clinics in each Region with 100%
information governance score (22/22)
National L3=149, L2=37
Wessex L3=5, L2=0
Anglia L3=10, L2=6
North Thames L3=27, L2=4
South Thames L3=24, L2=10
South West L3=12, L2=3
Cheshire & M'side L3=4, L2=0
Yorkshire L3=12, L2=6
Wales L3=10, L2=3
N. Ireland L3=5, L2=0
Northern L3=8, L2=0
Oxford L3=6, L2=1
North West L3=3, L2=1
West Midlands L3=8, L2=0
Trent L3=15, L2=3
Level 3, n=149
Level 2, n=37
Slide 39 of 59
38%
50%
80%
80%
17%
63%
0%
63%
70%
50%
33%
50%
42%
17%
40%
100%
40%
38%
33%
0%
33%
0%
25%
20%
0%
20%
33%
40%
60%
80%
Percentage of clinics meeting Standard
100%
KPI 7. Care pathways to Level 3 services
• 2 elements, documented evidence of explicit:
1. Agreed care pathways linking all providers of
services managing STIs in your area with Level 3
services
2. Level 3 leadership role for your area
Slide 40 of 59
KPI 7. Percentage of clinics in each Region with care
pathways to Level 3 services documented
National L3=149, L2=37
Oxford L3=6, L2=1
South Thames L3=24, L2=10
Wessex L3=5, L2=0
Wales L3=10, L2=3
Trent L3=15, L2=3
Cheshire & M'side L3=4, L2=0
North Thames L3=27, L2=4
Anglia L3=10, L2=6
Northern L3=8, L2=0
North West L3=3, L2=1
Yorkshire L3=12, L2=6
West Midlands L3=8, L2=0
South West L3=12, L2=3
N. Ireland L3=5, L2=0
Level 3, n=149
Level 2, n=37
Slide 41 of 59
50%
89%
80%
60%
0%
53%
100%
50%
41%
100%
40%
83%
38%
33%
100%
25%
100%
25%
25%
100%
0%
0%
100%
100%
92%
100%
20%
40%
60%
80%
Percentage of clinics meeting Standard
100%
KPI 7. Percentage of clinics in each Region with Level 3
leadership documented
National L3=149, L2=37
Wessex L3=5, L2=0
North West L3=3, L2=1
South Thames L3=24, L2=10
Oxford L3=6, L2=1
N. Ireland L3=5, L2=0
Northern L3=8, L2=0
Cheshire & M'side L3=4, L2=0
Wales L3=10, L2=3
West Midlands L3=8, L2=0
Anglia L3=10, L2=6
Yorkshire L3=12, L2=6
Trent L3=15, L2=3
North Thames L3=27, L2=4
South West L3=12, L2=3
Level 3, n=149
Level 2, n=37
Slide 42 of 59
62%
81%
100%
100%
0%
83%
83%
80%
100%
100%
75%
75%
70%
100%
63%
60%
58%
47%
41%
33%
0%
20%
40%
67%
83%
67%
50%
100%
60%
80%
Percentage of clinics meeting Standard
100%
KPI 8. Audit: annual participation & plan
• 2 elements, annual:
– Participation in a regional or national audit
– Completion of an audit plan
Slide 43 of 59
KPI 8. Percentage of clinics in each Region with evidence
of annual participation in audit
National L3=149, L2=37
West Midlands L3=8, L2=0
Wessex L3=5, L2=0
Wales L3=10, L2=3
South West L3=12, L2=3
Oxford L3=6, L2=1
North West L3=3, L2=1
North Thames L3=27, L2=4
Anglia L3=10, L2=6
Yorkshire L3=12, L2=6
Northern L3=8, L2=0
Trent L3=15, L2=3
South Thames L3=24, L2=10
N. Ireland L3=5, L2=0
Cheshire & M'side L3=4, L2=0
Level 3, n=149
Level 2, n=37
Slide 44 of 59
93%
49%
100%
100%
100%
100%
100%
67%
100%
100%
100%
0%
100%
50%
100%
17%
92%
33%
88%
87%
0%
83%
70%
80%
75%
0
0.2
0.4
0.6
0.8
Percentage of clinics meeting Standard
1
KPI 8. Percentage of clinics in each Region with evidence
of completion of annual audit plans
National L3=149, L2=37
West Midlands L3=8, L2=0
Wessex L3=5, L2=0
Anglia L3=10, L2=6
North Thames L3=27, L2=4
Wales L3=10, L2=3
South Thames L3=24, L2=10
Trent L3=15, L2=3
Oxford L3=6, L2=1
Northern L3=8, L2=0
South West L3=12, L2=3
Yorkshire L3=12, L2=6
Cheshire & M'side L3=4, L2=0
N. Ireland L3=5, L2=0
North West L3=3, L2=1
Level 3, n=149
Level 2, n=37
Slide 45 of 59
77%
43%
100%
0%
100%
100%
0%
93%
75%
83%
70%
67%
33%
90%
100%
67%
0%
63%
58%
0%
50%
17%
50%
40%
33%
0%
20%
40%
100%
60%
80%
Percentage of clinics meeting Standard
100%
KPI 9. Patient and Public Engagement (PPE) plan
• 4 components:
1. Documented PPE plan for 2010
•
•
Engagement with service users about services used, and
services they wished to attend
Engagement with the public, including non-users of STI
services, when
–
–
Any redesign or major service development is planned
Finding out why some groups don’t use services
2. Implementation, any part of PPE plan
3. Service user feedback arising from implementation
4. Response to service user feedback
Slide 46 of 59
KPI 9: Percentage of clinics in each Region with a
documented plan for PPE for 2010
National L3=149, L2=37
Wessex L3=5, L2=0
Northern L3=8, L2=0
Oxford L3=6, L2=1
Yorkshire L3=12, L2=6
South Thames L3=24, L2=10
North Thames L3=27, L2=4
Wales L3=10, L2=3
Anglia L3=10, L2=6
West Midlands L3=8, L2=0
South West L3=12, L2=3
Cheshire & M'side L3=4, L2=0
Trent L3=15, L2=3
North West L3=3, L2=1
N. Ireland L3=5, L2=0
Level 3, n=149
Level 2, n=37
Slide 47 of 59
62%
54%
100%
88%
83%
75%
33%
71%
63%
80%
100%
60%
0%
60%
17%
50%
50%
0%
50%
40%
100%
33%
100%
20%
0%
100%
20%
40%
60%
80%
Percentage of clinics meeting Standard
100%
KPI 9. Percentage of clinics in each Region with
implementation of PPE plans
National L3=149, L2=37
Wessex L3=5, L2=0
Northern L3=8, L2=0
Oxford L3=6, L2=1
North Thames L3=27, L2=4
South Thames L3=24, L2=10
West Midlands L3=8, L2=0
Yorkshire L3=12, L2=6
South West L3=12, L2=3
Wales L3=10, L2=3
Cheshire & M'side L3=4, L2=0
Anglia L3=10, L2=6
Trent L3=15, L2=3
North West L3=3, L2=1
N. Ireland L3=5, L2=0
Level 3, n=149
Level 2, n=37
Slide 48 of 59
54%
63%
100%
88%
83%
74%
71%
100%
100%
80%
63%
58%
33%
58%
0%
50%
0%
50%
50%
17%
47%
100%
33%
100%
20%
0%
20%
40%
60%
80%
Percentage of clinics meeting Standard
100%
KPI 9. Percentage of clinics in each Region with evidence
of service user feedback arising from implementation
National L3=149, L2=37
Wessex L3=5, L2=0
Oxford L3=6, L2=1
Northern L3=8, L2=0
Yorkshire L3=12, L2=6
North Thames L3=27, L2=4
South Thames L3=24, L2=10
West Midlands L3=8, L2=0
Wales L3=10, L2=3
Trent L3=15, L2=3
South West L3=12, L2=3
Cheshire & M'side L3=4, L2=0
N. Ireland L3=5, L2=0
Anglia L3=10, L2=6
North West L3=3, L2=1
Level 3, n=149
Level 2, n=37
Slide 49 of 59
66%
51%
100%
88%
100%
100%
75%
33%
74%
71%
63%
100%
80%
60%
0%
53%
50%
0%
67%
50%
40%
17%
0%
20%
40%
33%
40%
100%
60%
80%
Percentage of clinics meeting Standard
100%
KPI 9. Percentage of clinics in each Region with evidence
of response to service user feedback
National L3=149, L2=37
Wessex L3=5, L2=0
Oxford L3=6, L2=1
Northern L3=8, L2=0
North Thames L3=27, L2=4
South Thames L3=24, L2=10
Yorkshire L3=12, L2=6
Trent L3=15, L2=3
West Midlands L3=8, L2=0
South West L3=12, L2=3
Cheshire & M'side L3=4, L2=0
N. Ireland L3=5, L2=0
North West L3=3, L2=1
Wales L3=10, L2=3
Anglia L3=10, L2=6
Level 3, n=149
Level 2, n=37
Slide 50 of 59
51%
58%
100%
100%
100%
75%
67%
63%
100%
80%
58%
33%
53%
67%
50%
50%
0%
50%
40%
33%
0%
17%
0%
100%
30%
20%
30%
40%
60%
80%
Percentage of clinics meeting Standard
100%
Weaknesses
•
•
•
•
•
Definitions
Validation of questions used for data collection
Under-representation of Level 2 services
Reporting bias
Representation of smaller services
Slide 51 of 59
Summary National Performance, ranked by Level 3 KPI performance
HIV test uptake
Audit participation
48 hour access
PN index-reported 0.4 (London)
Audit plan
Sexual history
PPE patient feedback
PPE implementation
Level 3 leadership
KPIs
PPE plan
PPE response to patient feedback
STI/HIV risk assessment
Staff updating documentation
Care pathways
HIV test offer
Information governance
PN index-reported 0.6 (outside London)
Staff competency documentation
Results received by clincians within 7 days
49%
51%
43%
43%
59%
66%
51%
63%
54%
62%
62%
54%
58%
51%
56%
43%
54%
65%
50%
50%
43%
50%
38%
43%
39%
40%
62%
39%
57%
0%
Level 3
Slide 52 of 59
Level 2
94%
93%
70%
20%
40%
60%
86%
85%
77%
77%
81%
89%
80%
Percentage of clinics meeting KPI standard
100%
Key messages
• BASHH KPIs required further definition to
allow auditing
→BASHH Clinical Standards Unit
• KPIs are achievable by both Level 2 & Level 3
services
• Variable performance against all KPIs across
regions
• Areas for improvement
Slide 53 of 59
Areas for improvement
• Information governance
– Training
– Policies
• Time to test results
– Laboratories/clinic systems
• Documentation of staff competency
– Further national work in progress to describe
competencies
• PN at 0.6 level (verification)
– “Dedicated ” PN time
– ePN (Ann Sullivan)
Slide 54 of 59
Making the best use of audit results- 1
• Presenting/discussing in clinical meetings
– Interventions needed to improve practice
– Implementation
• Relating change to individual and team practice
– Recording staff competency and updating
– Structures and processes to follow up and document partner
notification outcomes
– Re-design of case note or EPR recording
– Improve uptake of HIV testing
– Improve access to test results
– Care pathways to connect all STI management providers to Level
3 services
– Planning ahead for audit work
Slide 55 of 59
Making the best use of audit results- 2
• Planning change management
– Managers
– Commissioners
– Meetings: clinical, operational, clinical governance, staff
meetings, meetings
• Patients involvement in change management
• Using the audit exercise for appraisal, revalidation
and performance review
Slide 56 of 59
Change Champions
• Collect examples from clinics where the STIMS
Audit data was used
– To improve practice
– How this was achieved
– Share learning with other clinics, whose patients
might also benefit
Slide 57 of 59
Acknowledgements
• Funding: Sexual Health and HIV, Department of
Health
• Planning: BASHH National Audit Group
• Participation:
– Level 2 services: Andrea Duncan Sexual Health and HIV
Programme Manager, Department of Health
– Level 3 services: BASHH Regional Audit Chairs
– All clinics
– Service managers
• Data collection and aggregation: Hilary Curtis
Slide 58 of 59
Acknowledgements: BASHH Regional Audit Chairs &
Members
Chair Hugo McClean
Vice Chair Chris Carne
Hon Sec Ann Sullivan
Director of Development Anatole
Menon-Johansson
BCCG Representative Phil Kell
BHIVA Representative Alison Rodger, Ed
Wilkins
Scotland Daniel Clutterbuck
Wales Helen Baley, Sarah McAndrew,
Carys Knapper
Northern Ireland Say Quah
Anglia Raouf Moussa
Cheshire & Mersey Ravindra Gokhale
Slide 59 of 59
Essex Gail Crowe
North Thames Ann Sullivan, Alan Smith
Northern Sarup Tayal
North-West Ashish Sukthankar
Oxford Gill Wildman
South East Thames Cindy Sethi
South-West Zoe Warwick
South-West Thames Steven Estreich
Trent Jyoti Dhar
Wessex Neelam Radja, Leela Sanmani
West Midlands Sashi Acharya
Yorkshire Amy Tobin-Mammen
Co-opted Members David Daniels,
Nicola Low, Lindsay Emmett