BOS CLINICAL EFFECTIVENESS
COMMITTEE
NATIONAL SURVEY OF AUDIT ACTIVITY
Forms returned covering the period 2005
A survey of the regional orthodontic audit groups across the UK has taken place using the
downloadable BOS audit survey forms 1, 2a and 2b. The survey established what projects
were ongoing (Form 2a) and what were completed (Form 2b) during the period 2005.
This should provide a useful reference for planning future audits and for BOS members
to be aware of audit activity nationally. Each regional coordinator will doubtless be
pleased to provide more details of individual audits upon request.
INDEX
Section A) Ongoing projects for survey period
Pages 2-6
Section B) Completed projects for survey period
Pages 7-13
Table 1. Summary of responses by section
Page 14
Graphical representation of responses
Pages 15-16
.
1
A) ONGOING PROJECTS FOR SURVEY PERIOD
EASTERN
Title. Informed Consent in Orthodontics
Aims & standards. 80% correct answers to a range of questions drawn from the
November 2001 DoH guidance on Consent, as well as for 80% of all patients to have
signed an acceptable consent form before starting treatment.
Title. Dental Unit Waterline Disinfection (DUWL)
Aims & standards. To achieve EU potable (drinking) water quality standards for the
water emanating from the hospital DUWLs. To have less than 100 colony forming units
per ml of water (<100 cfu / ml)
Title. Accuracy of pre-surgical cephalometric predictions using OPAL
Aims & standards. To examine the discrepancies between the OPAL predictions and the
final post-operative outcomes for 6 cases.
MERSEY
Title. Audit of attendance at regional audit meetings
Aims & standards. To monitor overall levels of attendance by hospital based staff at
Regional Orthodontic Audit Meetings and also to contribute to establishing a national
minimum standard. In the absence of definite national standards we would aim to
achieve a minimum of 75% attendance levels annually.
Title. New patient satisfaction
Aims & standards. Survey to exploring how patient satisfaction relates to a variety of
variables involved during the patient journey.
Title. Audit on the use of headgear in Liverpool Dental Hospital
Aims & standards. To assess if the orthodontic department is complying with British
Orthodontic Society guidelines on the safe use of headgear
NORTHERN IRELAND
Title. Audit of Surgical outcomes
Aims & standards. Benchmark surgical outcomes to published national standards
2
NORTHERN IRELAND cont.
Title. Audit of replacement retainers
Aims & standards Does an advice sheet reduce the frequency of loss or fracture of essix
retainers
NORTHERN REGION
Title. Failure to attend appointments
Aims & standards. There should be 100% non-failure of appointments of patients in
treatment.
Title. Discontinuation rates of patients in orthodontic treatment.
Aims & standards. There should be no failure of completion of treatment.
Title. Reality Orthodontics
Aims & standards. Present consecutive slides of patients in treatment, warts and all, of
named patients.
NORTH THAMES
Title. CNST trust wide audit
Aims & standards. To monitor compliance of documentation against CNST standards and
Trust policies
Title. CNST departmental pink notes how do they compare?
Aims & standards. To monitor compliance of documentation against CNST standards and
Trust policies
Title. Patient satisfaction survey
Aims & standards. To get patient feedback regarding their experience of treatment within
the department.
Title. Audit of registrar attendance on treatment clinics
Aims & standards. Establish attendance on treatment clinics. Establish number of patients
seen per treatment clinic. Registrars should be on every clinic as per their timetable.
3
OXFORD
Title. Orthognathic clinic – patient satisfaction
Aims & standards. To assess patient understanding for attendance at Joint orthognathic
clinic at the start of treatment. To assess their experience of attending the clinic.
SCOTLAND
Title. Reasons for clinicians running late
Aims & standards. All patients should be seen on time
Title. Availability of Pre and Post treatment study models
Aims & standards. To assess the availability and quality of patient study models
SOUTH EAST
Title. New Patient Referrals with Impacted Canines:
Aims & standards. To determine the likely patient numbers for a prospective audit on
methods and efficacy of combined orthodontic and surgical treatments for impacted
canines
SOUTH WEST
Title. Care pathways for palatal canines
Aims & Standards. To establish the length of time from initial referral to ‘in-treatment’ of
patient with palatal canine/s. No greater than 6 months from referral to treatment
Title. Audit of Orthognathic outcome compared with planned outcome
Aims & standards. Protocol being written
Title. Audit of fixed retainers
Aims & standards. Protocol being written
Title. Audit of success of sleep apnoea appliances
Aims & standards. Protocol being written
Title. Patient satisfaction
Aims & standards. Protocol being written
4
WALES
Title of project. Audit of Impacted Canines
Aims & standards. The aims are to assess the interceptive success of extracting deciduous
canines in improving the eruptive pathway of palatally impacted permanent canines and
to assess the success of open exposure of the canines compared to alignment with gold
chain.
Title. Audit of Casual Orthodontic Attendance at the University Dental Hospital, Cardiff
Aims & standards. Indicate % of unscheduled visits in a given period. Less than 5% of
visits to an orthodontic clinic should be unscheduled (RCS Eng). Outline basic
demographics. E.g. male/female, age, patient adherence to attendance times, patient
waiting times, treatment times, type of problem.
Title. Audit to assess compliance with guidelines for dataset of orthognathic surgery
cases.
Aims & standards. The aims of the Audit are to assess the compliance with the
BAOMS/BOS Guidelines on minimum dataset for clinical records for patients
undergoing orthognathic surgery and to assess the quality of the records taken.
Title. Satisfaction with Orthognathic Surgery
Aims & standards. To assess both the patient’s and clinician’s satisfaction with the
outcome of orthognathic surgery.
WESSEX
Title. Regional new patient referral audit
Aims & standards. An “ideal” new patient referral should meet six criteria. Arbitrarily, if
a referral meets four of the six criteria it would be considered “appropriate”. Gold
standard:- 95% of referrals should be “appropriate”.
WEST MIDLANDS
Title. Par scoring
Aims. To collect all par scores for local and national comparisons
5
YORKSHIRE
Title. IOTN and PAR accuracy: a local comparative audit
Aims & standards: The aim is to test IOTN and PAR scoring inter-operator
reproducibility and intra-operator reliability of the five Leeds Orthodontic Technicians (3
calibrated & 2 non-calibrated).
Title. Functional Appliance Information: “Are We Getting Through To Our Patients?”
Aims & standards: Multi-centre questionnaire based audit is to assess patients
understanding of functional appliances and determine any differences between patients
understanding of treatment between nurse-led and non nurse-led clinics.
Title. Regional outcomes of palatal canine exposures
Aims & standards: This regional audit is to determine the success of exposing and
orthodontically aligning palatally impacted maxillary canines in each orthodontic
department in the Yorkshire region. Gold Standard: 90% of exposed canines should have
good or acceptable treatment results.
Title. The prevalence of demineralisation following fixed appliance therapy.
Aims & standards: The aims of the audit are to determine the prevalence of
demineralisation during fixed appliance therapy and also examine other patient
parameters e.g. OH and length of time in appliances. The “Gold Standard” is that no
more than 25% of patients will suffer more than one new white spot lesion following
fixed appliances.
Title. The Accuracy of Orthognathic Surgical Movements
Aims & standards: The aim of this audit was to compare the planned orthognathic
surgical movements against the actual movements achieved. The Gold Standard is that
90% of all maxillary and mandibular movements should be within 1mm of the planned
surgical movements both vertically and antero-posteriorly (A-P).
Title. Patients understanding of consent
Aims & standards: The aim of this audit is to determine whether or not patients receive
informed consent. The Gold Standard is that 90% of patients should understand what
their treatment will involve, the length of their treatment, risks of treatment and their role
in their treatment.
Title. Consultant PAR: Regional Audit
Aims & standards: To assess the average consultant % PAR reduction. The Gold
Standard being that the mean % PAR change should be greater that 70%.
Title. Secret audit: St Luke’s patient waiting times
6
Aims & standards: To audit how long patients wait to be seen in relation to their
appointment time. The Gold Standard is that 90% of patients should be seen within 20
minutes of their appointment time.
7
B) COMPLETED PROJECTS FOR SURVEY PERIOD
EASTERN REGION
Title. An audit of impressions and study models
Aims & standards. To assess quality of impressions taken by orthodontists and DCPs, as
well as the study models returned.
MERSEY REGION
Title. Reply letters following orthodontic consultations: An audit of GDP satisfaction
Aims & standards. Assess the content of consultant orthodontists reply letters, in
particular the GDPs opinion regarding the length and appropriateness of the information
included.
Title. Clinical Photography Audit
Aims & standards. To assess the quality of clinical photography in the orthodontic
department. Liverpool University Dental Hospital. Standards Based on MOrth
requirements
NORTH MANCHESTER AND LANCASHIRE
Title. Treatment Information and Patient Consent : Regional Audit Previous single unit
study completed
Aims & standards. To determine if there was a written consent form in every patient’s
notes. Did we give patients enough information about their treatment. Did the patients
understand the treatment and risks involved. Were the patients left with any important
unanswered questions. Audit standard was 90% compliance
Title. Waiting List Management Audit
Aims & Standards. Every patient bypassing an Orthodontic Waiting List (OWL) means
another patient waits longer. Aim is therefore to assess the probity of reasons for ‘fasttracking’. And its potential impact on OWL management
Valid reasons for bypassing OWL are: Joint speciality surgical cases. Ectopic or
impacted teeth. Osteotomy patients. Patients distressed about malocclusion. Growing
patients with increased overjets. Cases ‘Transferred in Treatment’
Audit standard is that 90% of cases fast-tracked should meet the above criteria. Those
that do not should have the reason stated.
8
NORTH MANCHESTER AND LANCASHIRE cont.
Title. Orthognathic Surgery Treatment Times : Retrospective Regional Audit. Previous
single unit study carried out
Aims & standards. To audit the time taken between each stage of treatment in cases
involving orthognathic surgery. To improve the quality of information given to
orthognathic patients regarding waiting times between appointments & overall treatment
times.
Audit Standards. Referral date to initial consultation = 13 weeks. Initial consultation to
joint clinic = 13 weeks. Joint clinic to fixed treatment = 1-6 months. Fixed treatment to
pre-surgical j.c = 18-24 months. Pre-surgical joint clinic to surgery = 1-6 months.
Surgery to debond = 3-6 months. Total duration of fixed appliance = 24-36 months
Title. Bonded Retainer Failure Rate Regional Audit. Previous two-unit study completed
Aims. Retrospective study of patient records; To audit the failure rate of bonded
orthodontic retainers in the immediate 12 month follow-up period for retainers fitted
during 2002/2003.
Audit Standard. ≤ 10% initial and overall bond failure rate. ≤ 10% individual bond
failure rate.
NORTH THAMES
Title. Length of orthodontic treatment pre/ post orthognathic surgery
Aims & standards. Retrospective study. To establish the length of orthodontic treatment
prior to and after orthognathic surgery. All patients undergoing orthognathic surgery 1st
Jan -31st Dec 2004.
Title. Patient attendance rates
Aims & standards. Prospective look at the reason for DNA within the department.
Questionnaire based.
Title. Quality of our record keeping
Aims & standards. To compare the standard of record keeping in line with Trust policies
and guidelines to good clinical practice.
NORTHERN IRELAND
Title. Audit of Laboratory Activity
Aims & standards To assess the amount of orthodontic lab work undertaken in a two
month period
9
NORTHERN IRELAND cont.
Title. Participation in National Outcomes Audit for patients born with UCLP (6
consecutive cases)
Aims & standards –not stated
Title. Patients treated by SHO
Aims & standards -not stated
Title. Instrument Usage Audit
Aims & standards. To determine type and number of instruments used on a daily basis
Title. Participation in National Audit on the use of outcome monitoring
Aims & standards –not stated
Title. Quality assessment of OPTs. Repeat of 2002 audit.
Aims & standards. To assess quality of OPTs taken in Altnagelvin hospital. Criteria
based on “essentials of dental radiography and radiology” E. Waites.
Title. Audit of written reporting of orthodontic and OMFS radiographs
Aims & standards. 100% reporting
Title.Audit of 22 consecutively completed cases personally treated by consultant
orthodontist.
Aims & standards. 75% of cases should exhibit a reduction in PAR score>70% with 3%
or fewer having a reduction in PAR of <30%
Title. Audit of treatment need of cases undergoing orthodontic treatment with the
consultant
Aims & standards. To use IOTN to assess treatment need of patients undergoing
treatment with the consultant and to compare to last year
Title. Audit of referrals to the Hospital Orthodontic Service.
Aims & standards. To identify source of referrals over a three month period and compare
to last year
Title. Comparison of pre-treatment need and complexity scores for orthodontic patients in
three different clinics using ICON
Aims & standards. To compare treatment need and complexity of patients attending for
orthodontic treatment at three different orthodontic clinics (two hospital, one specialist
practice.
10
NORTHERN REGION
Title. Supernumerary teeth and impacted incisors
Aims & standards. Aim was to determine gold standards from the literature and compare
regional performances against this determined gold standard.
Title. Audit of 100 PAR cases
Aims & standards. Last 100 consecutive cases finished by H.H.G.
Title of project. Length of time patients are in treatment for osteotomy
Aims & standards. There should be seamless progression of treatment from completion
of orthodontic treatment to progression of surgery.
OXFORD
Title. An Audit of treatment time for orthognathic patients in Oxford Trust
Aims & standards. To assess total treatment time for orthognathic patients (assessed over
a year)
SCOTLAND
Title. Quality of result and extension of treatment time of transferred cases
Aims & standards To assess whether transferred cases between operators suffer increased
treatment time and reduced quality of result compared with cases treated by single
operator
Title. Complexity Outcome and Need of cases treated by the Consultant Orthodontic
Service in Scotland
Aims & standards. Existing standards- 90 %+ of cases should be IOTN (DHC) 4 or 5
75% + should be difficult or v difficult using ICON. 75% + should have a PAR reduction
> 70 %.
Title. Records for orthodontic clinics
Aims & standards. All patient records available at every visit ie S/Ms, X-Rays and photos
Title. Model box audit
Aims & standards. To identify unwanted materials- archwires and appliances
11
SOUTH EAST
Title. Compliance with the minimum dataset for orthognathic treatment
Aims & standards. To determine our compliance with the dataset from BAOMS and
BOC
SOUTH WEST
Title. Prospective Audit on the Success Rate of Functional Appliances
Aims. All overjets should be reduced by half after 6 months. No patients should
discontinue treatment
Title. Casenote Audit
Aims & standards. BOS guidelines used (13 criteria). 100% should be achieved for all
criteria.
Title. Audit of archwire breakages
Aims. 1) To assess whether archwire problems were still the primary cause of
unscheduled orthodontic attendances. 2) To reduce unscheduled attendances due to
archwire problems.
Title. Are radiographs requested in the orthodontic department appropriate and justified?
Aims. The aim of this audit is to re-assess the appropriateness of requests for radiographs
in the orthodontic department at a new patient consultation.
SOUTH WEST THAMES
Title. A baseline audit of the availability of cleft lip and palate records in South Thames.
Aims & standards. 100% Records to be available for extra oral/Intra oral photograph oral
study models. 5 years, 10 years, 15 years
TRENT
Title. Pilot audit of Yellow adverse reaction reports forms
Aims & standards. To assess how these would be used if on line forms would be more
accessible
12
WESSEX
Title. Cross infection control
Aims & standards. Compliance with BDA advice sheet A12
Title of project. Effectiveness of pre-orthodontic oral hygiene clinics
Aims & standards. To prospectively assess differences in OH scores during the fixed
orthodontic appliance phase of treatment between patients previously seen on a formal
OH instruction clinic and those given routine OH advice by the treating orthodontist.
The baseline pre-treatment OH minimum standard required for fixed orthodontic
appliance treatment was considered to be 80% clean plaque score 1. The oral hygiene
score during active treatment should be no greater than 30-50% plaque score using the
specific Orthodontic Plaque Index (OPI) 2.
Title. An audit of bonded molar tube usage and reliability.
Aims & standards. A X sectional study to determine 1) if there had been an increase in
use of bonded molar tubes 2) failure rate for molar tubes 3) which tubes most likely to
fail and 4) if safety measures were taken to prevent loss of tubes. It was intended to use
this audit to set a gold standard for molar tube failure and then to carry out a prospective
audit to look at molar tube usage and breakage/complication rate
WEST MIDLANDS
Title. Consent for orthodontic procedures and clinical photographs
Aims & standards. To audit the proportion of cases conforming to UHNS consent
policies for treatment and photography.
Title. Audit of referral patterns for one consultant UHNS.
Aims & standards. To assess the type of referrer and suitability of referral for hospital
assessment and treatment.
Title. An Audit of Photography and Photographic Consent Procedures in Orthodontics in
the West Midlands Deanery
Aims & standards. 1) BOS Guidelines 2) DoH Guidelines 3) Institute of Medical
Illustrators Guidelines
Title. Retention Clinic Attendance
Aims & standards. To investigate retainer clinic attendance patterns and frequency with
which patients bring retainers to appointments (Standard = 100%)
Title. Bracket failure
Aims & standards. Assess bracket failure rate and compare to published norms
13
WEST MIDLANDS cont.
Title. An audit of radiograph reporting
Aims & standards. To compare departmental reporting rate to national guidelines / BOS
Gold Standard. IR(ME)R 2000:- “The practitioner or referrer must enter a clinical
evaluation of the radiograph into the patients notes”. 100% of radiographs should be
reported.
Title. An audit of correct band selection.
Aims & standards. To assess band blank and gauge in helping choose a correct band size
for patients. Gold standard is always correct choice at first try. Pilot study score = the
difference the size of the first band chosen and that of the actual size fitted e.g. try size
38 first but fit size 33 = score 5.
Title. An audit of consultant treatment waiting times and waiting list management in the
West Midlands.
Aims & standards. To review the waiting times and types waiting lists held by
consultants in the West Midlands
Title. An audit of use of an oral Vibrating Stimulation to reduce post-bandup pain.
Aims & standards. To assess effectiveness of the bitefork in reducing initial pain after
bind up. A recent paper in the JCO reported on a vibrating soft bite fork the like of which
was used in this study, 38 patients divided into two groups.
YORKSHIRE
Title. Correspondence audit at Leeds Dental Institute
Aims & standards. This aim of this secret audit at the LDI was to assess the level of
correspondence between the orthodontists and GDP’s, both at initial assessment and
during treatment. The gold standard was that each patient should have a letter sent to
their GDP following the initial examination, at the start and end of active treatment, and
at discharge.
Title. DNA rates for review appointments at St Luke’s Hospital.
Aims & standards. The audit specifically assessed DNA rates related to consultant review
appointments made six months or more in advance. 100 review patients were collected
prospectively.
14
Table 1. Summary of responses for recorded projects
Audits
Ongoing
Completed
Total
Type of audit1
(More than one response possible)
Waiting times
NHS targets
Clinical technique
Outcome audit
Clinical guidelines and pathways
Educational
Patient satisfaction
Proceedural issues
Other
Stage in audit cycle1
Initial data collection
Institute changes
Complete the cycle/spiral
Other
37
51
88
8
18
12
23
17
4
7
10
10
49
23
8
7
Collaboration2
Multidisciplinary
With different care providers
Average number of units involved
22
19
4
Note1 Some audits ticked by regional coordinators as belonging to more than one ‘Type
of audit’ or ‘Stage of audit’
Note2 Numbers in the ‘Collaboration’ reporting box are less than total report forms
submitted. Remaining audits ticked as ‘No’ or not ticked.
15
S
t im
es
Cl
in
ic
37
51
18
12
8
4
16
7
10
O
th
er
25
20
15
10
5
0
ta
rg
et
s
al
te
ch
ni
qu
O
ut
e
co
m
e
Cl
au
in
ic
di
al
t
gu
id
el
i. .
.
Ed
uc
Pa
at
io
tie
na
nt
l
sa
t
isf
Pr
ac
oc
...
ee
du
ra
li
s.
..
NH
W
ai
t in
g
Figure 1. Audits ongoing and completed 2005
Ongoing
Completed
Figure 2. Type of audit
23
17
10
Figure 3. Stage in audit cycle
60
50
49
40
30
23
20
8
7
Complete the
cycle/spiral
Other
10
0
Initial data
collection
Institute changes
17