A Significant Event Analysis and Audit Toolkit for Dental Professionals 1 Contents Introduction to Clinical Audit and Significant Event Analysis Using SEA as an adjunct to audit Individual SEAs and audits Practice / team SEAs and audits Significant Event Analysis What is Significant Event Analysis Analysing Significant events The 7 steps of SEA Producing a SEA report Clinical audit Clinical audit - overview To carry out an audit Steps 1 to 13 The audit planning tool Writing an audit report Ideas for audit topics Illustration of possible audit topics, professional groups, sample size and CPD hours How to apply for Audit and SEA project approval Paperwork and claims forms Application for DCP Audit Project approval Application for DCP & dentist SEA approval How to apply for audit project and SEA approval How to claim Clinical Audit Allowance GP216A (rev 11/1o) Annex 1 to GP 216A (rev 4/11) Annex 2 to GP 216A (rev 4/11) GP216B rev (4/11) Annex 1 to GP 216B (rev 4/11) Annex 2 to GP 216B (rev 4/11) GP 217 (rev 11/10) Frequently asked questions NES regional audit administrators / audit contacts by region Useful links and sources of gold standard evidence Examples of audits 2 Significant Event Analysis and Clinical Audit an introduction Significant Event Analysis (SEA) and Clinical Audit are recognised as important elements of Clinical Governance which contribute to Patient Safety in Dental Primary Care. In order to maintain a high standard of patient care it is important to continually review performance and strive to improve. Clinical audit has been a requirement for dentists under the NHS terms of Service since 2001 and is also recognised as verifiable Continued Professional Development (CPD) by the General Dental Council (GDC) for all registrants. The aim of this site is to give all dental professionals access to the information and people they need in order to begin, or continue, to use clinical audit or SEA and to make improvements to both their practice and patient safety. Scotland has a network of personnel working to facilitate your participation in SEA and Clinical Audit. We would like to encourage you to take part in whatever type of SEA/audit project you feel is relevant to your role. The ideas and topics included in these pages are the tip of an iceberg and are not prescriptive. Both SEA and Clinical Audit are of more value if based on a real need for improvement which has been identified in day to day work or as the result of appraisal. Many sources of guidance are available to dental professionals, including the Scottish Dental Clinical Effectiveness Programme (SDCEP), the Scottish Intercollegiate Guideline Network (SIGN), the Faculty of General Dental Practitioners (FGDP) and, in Scotland, the entire evidence base has been summarised and is available to all dental professionals on http://www.scottishdental.org The evidence based guidance produced by (SDCEP) is particularly relevant to all who work in Primary Dental Care and is user friendly and regularly updated. Using SEA as an adjunct to audit Significant events happen routinely in dental practice, often highlighting areas where improvement could be made in practice systems, however because of lack of time, commitment, or knowledge, opportunities to correct deficiencies are missed. Carrying out SEA will help to identify and prioritise the underlying issues which have contributed to the significant events, allowing you to undertake a structured analysis of the event(s). By undertaking a few SEAs you will become to understand the importance and role of SEA as a non-threatening flexible tool for reflective learning. By producing a smooth introduction of regular team based discussion and analysis, improved patient care will follow. Well designed SEA and Clinical Audits can be counted as verifiable CPD. It should be noted that while NES intends to deal with SEA reports in strict confidence, please be aware that where there are overwhelming clinical or ethical reasons to do so, NES may be obliged to take further action which may include sharing information from the SEA report with relevant professional bodies. 3 Individual SEAs and Audits Many dental professionals wish to monitor the quality of certain aspects of their role within the team, such as the quality of radiographs or equipment care. This can be undertaken as an individual audit and can be used as a driver in developing personal development plans. NES tutor / audit facilitator support is available to help the planning and execution of this type of audit. Practice/Team SEAs and Audits This type of audit is more suitable for dental professionals who are confident at leading a project within their own practice. The project may be limited to DCPs or dentists but would be of greater benefit to the whole practice if extended to incorporate the entire dental team. An example of team audit might be testing staff knowledge and skills before and after “in practice” decontamination training thereby linking a practice based training exercise to a demonstrable increase / improvement in knowledge and clinical practice. Using this type of audit to cover other issues relating to Clinical Governance and Best Practice would encourage dental team members to develop their individual knowledge and skills further and to become more reflective. Facilitation and support is also available from NES in the planning and execution of this type of audit. Significant Event Analysis What is Significant Event Analysis Many incidents happen in dental practices that would be considered as significant. An SEA can be regarded as “….any event thought by anyone in the team to be significant in the care of patients or the conduct or the practice.” These events are often positive and create a good experience for the patient. The team have worked to a high standard and this has been reflected in patient care. Practice teams can look at these positive events and identify what individual factors came together on these occasions to produce the positive outcomes. Unfortunately, some events are negative, these areas can also benefit from the practice undertaking a SEA. SEA is a type of audit which takes an individual event and gives every person involved a chance to reflect and establish what happened, why, and what can be learned from this. The essence of SEA is to identify and learn from the experiences. The process can be used to: Highlight significant incidents and how they contribute to the performance of the practice As an opportunity to identify what it was the team did well or improve the role of the dental team - reflecting and learning from important events Discuss the event and each individual’s part in it Take appropriate action by identifying training needs of team or individuals, introducing the necessary changes And, of course, enhance patient safety 4 Analysing Significant Events Simply acknowledging and discussing a significant event among colleagues after it happens is not enough; it is likely to recur if that is all you do. Using the seven step approach to identifying and investigating events can highlight care or methods of practise that could be improved. The 7 Steps involved in Significant Event Analysis Step Information Requirements 1 Identify the event Record the event Identify significant events for analysis and highlight these at a suitable forum e.g. dedicated SEA or team meetings. Enable staff to routinely record significant events using a log book, a special pro-forma or other suitable method. 2 Collect information Collect as much factual information as possible for each event including written records, and the thoughts and opinions of those directly and indirectly involved. This may include patients or relatives or health care professionals based outside the practice. Have all information ready to take to a meeting. Collate information 3 Arrange a meeting to discuss events Discuss with the person responsible for organising the meeting that you wish to have SEA added to the agenda. Ideally it would be the individual has collected the information about the event should introduce it at the meeting. See note below to optimise the meeting. 4 Meet and undertake a structured analysis Using these questions consider the relevant information and, as a team, agree how to improve performance what happened? why did it happen? what has been learned? what do we need to change? See note below on the structured analysis. 5 Implement changes and monitor progress The team member taking the SEA forward agrees the changes, facilitates the implementation, monitors progress, makes the SEA an agenda item on next practice meeting and reports to the meeting. Use form “Application for SEA Approval” for submission of SEA if Clinical Audit allowance is being claimed. 6 Writing up the SEA Record the change. If appropriate create a new protocol to ensure consistency for the new process / approach. Write-up the event analysis, once change has been agreed and implemented. This provides documentary evidence that the event has been dealt with. It is good practice to attach any additional evidence (e.g. an amended protocol) to the report. The report should be written-up by the individual who has the greatest knowledge of the event or who led on the event analysis. 7 Seek external comment / educational feedback Submit SEA report for external review and gain sign off for Clinical Audit Allowance purposes as above in step 5. See note below on external comments and feedback. 5 The meeting. The meeting should be conducted in an open, fair, honest and non-threatening atmosphere. A note of the meeting should be taken and circulated. A major attraction of the significant event analysis method of audit is that it is a team based activity that should be undertaken in a non-threatening and no-blame environment. The traditional method of SEA stresses that there should be no hierarchy - seniority and rank are abandoned at the door of the meeting - with all team members being treated courteously and given time to contribute their viewpoint. This must be made clear to everyone at the outset of the meeting. If any team member fears reprisal or disciplinary action for highlighting a significant event then the process is unlikely to work. The person you choose to facilitate a significant event meeting or to take responsibility for an event analysis again depends on practice dynamics and staff confidence. One suggestion is for the individual with the greatest knowledge of the event to assume this role if they are happy to do this. This may be straightforward for many experienced staff, but daunting for other members of the practice. The practice manager could facilitate the process initially, especially for administrative events, and then support and encourage other staff to take charge of future analyses. For many practices adopting this approach may involve a complete change in culture, so be prepared to be patient and to experience some early teething problems. The structured analysis. The focus of the meeting should be on establishing exactly what happened and why it happened, with the main emphasis being on learning from the event and changing behaviours, procedures, or systems, where appropriate. If this is the case, the purpose of the analysis is to minimise or prevent the chances of the event recurring. One method of analysing a significant event in a structured manner is by answering, indepth, the following four questions: • What happened? Gather all the facts relating to the significant event (including relevant dates, times, and people or organisations involved) from those directly and indirectly involved. This allows you to establish a clear and full picture of what happened. It is important to consider what the impact or potential impact of the event was for the patient, the team and/or others. You should gather as much of this information as possible before the event is discussed at the meeting. But time and staff availability are factors here and it is not always immediately clear why an event happened, so you may need to tease this out at the meeting. • Why did it happen? At the meeting, establish all of the main and underlying reasons why the event occurred. Consider, for instance, the professionalism of the team, the lack of a practice system or a failing in an existing system, lack of knowledge or the complexity associated with the event. For example, an important radiograph had been lost. But why was it lost? It was lost because it was pinned to the outside of a case note, which was subsequently picked up and filed away by an unsuspecting member of staff who did not notice the radiograph falling off the notes. The practice did not have an adequate system for filing of radiographs. • What has been learned? Highlight any learning issues you and/or the practice experience. You should be able to demonstrate that reflection and learning have taken place on an individual or team basis. For example, it may be related to a training need or to personal learning issues concerned with clinical techniques, or administrative procedures. It could also reflect a learning experience (good or bad) in dealing with patients, colleagues or other organisations. 6 • What have you changed? With many significant events, you will need to make a change to improve the provision of care or minimize the risk that a similar event will occur, or both. Consider, for instance, if an existing protocol needs to be updated or a new one developed, or if members of staff require additional training. If so, you need to ensure that affirmative action is to be taken rather than simply discussing what changes you would like to see implemented or documenting a wish list of actions that have no real prospect of being carried out. A member of staff should be designated to lead on the change and report back on progress at future meetings. It may highlight a need to include an issue in your Practice Development Plan (PDP). On some occasions it may not be possible to implement change. For example, the likelihood of the event happening again may be very small, or change may be out of your control. If so, clearly document why you have not taken action. External comment / educational feedback. Seek external feedback on the standard of the event analysis undertaken once it has been written-up. This is because research has repeatedly shown that around one third of event analyses are considered to be unsatisfactory, mainly because the team has failed to fully understand why the event happened or failed to take necessary or appropriate action to prevent recurrence. Sharing the event analysis with others allows the opportunity for objective comment on your event analysis. Producing your SEA Report Use the seven step approach (above) to report your SEA. Be sure to incorporate the following: Date of event. Date of meeting. SEA Lead. 1. What happened? 2. Why did it happen? 3. Reflection and learning? 4. What action has been taken? Remember, the report should reflect an in-depth analysis of each of these areas rather than a superficial attempt to simply describe the event. It is good practice to keep the report anonymous so that individuals and other organisations cannot be identified. No patient identifiable information should be included in the report. The paperwork to be used to write your SEA report is “Application for SEA Approval” In the case of dentists, who wish to claim Clinical Audit Allowance, the completed report should accompany the post - SEA submission to the regional postgraduate office. 7 Clinical audit Clinical audit can be described as a cycle or a spiral, see figure below. Within the cycle there are stages that follow the systematic process of: identifying the area to be improved (the problem or issue) establishing best practice & setting criteria and standards observing what you are currently doing and collecting data comparing against the criteria and standards taking action to improve care – implementing change monitoring to sustain improvement and re audit As the process continues, each cycle aspires to a higher level of quality of performance. The process requires you to identify areas for improvement, implement action plans to rectify or improve patient care or performance and then to re-audit to ensure that these changes have an effect. To carry out an audit. Using the audit planning tool on page 12 and following steps below to guide you through the audit process. If you require assistance with regard to any stage of the audit process or to determine if your project meets the criteria for verifiable CPD then use the audit planning tool and email an outline of your audit as a word document it to the audit administrator at the regional postgraduate office. The audit administrator will direct you to the appropriate person to assist you. Stage 1: Identify the problem / issue or topic This stage involves the selection of a topic or issue to be audited, which might have resulted from a SEA or from some part of your practise that you felt should be improved. Selection of an audit topic is influenced by factors including: where clear national standards and guidelines exist e.g. SDCEP guidance areas where problems have been encountered in practice outcomes of a SEA 8 where there is a clear potential for improving patient care areas of high volume, high risk or high cost, in which improvements can be made for the benefit of the practice Stage 2: Define the criteria & standards Define the overall purpose of the audit, either as what should happen as a result of the audit or what question you want the audit to answer This should be written as a series of statements or tasks on which the audit will focus. Collectively, these form the audit criteria. The criteria are explicit statements that define what is being measured and represent elements of care / performance that can be measured objectively. The standards define the aspect of care / performance to be measured, and should always be based on the best available evidence. Criteria are factors which can be measured. For example, “liquid soap, in single dose cartridge dispensers, is available at all hand washing sinks” , “there is a, plugless, handwashing basin with elbow / wrist or sensor operated mixer taps” or “the radiographic distance from the end of the root filling to the apex is less than 0.5mm”. Standards are the threshold of the expected compliance for each criterion or the number of times the criteria are achieved (usually expressed as a percentage). For the above example an appropriate standards would be “85% of root fillings should end within 0.5 mm of the apex”. If you require help identifying appropriate criteria or standards please contact your regional postgraduate office or audit facilitator. http://www.scottishdental.org is an excellent site with links to a large number of standards documents e.g. SDCEP and SIGN guidance. Stage 3: Data collection Agree the data to be collected. Do not be tempted to collect too much data. Remember this should be a small, stand alone audit not a research project. To be sure that the data collected is fit for your purpose, follow the steps below: define inclusion and exclusion criteria i.e. exactly what information you do and do not want to collect do not collect unnecessary data decide whether you will collect the data retrospectively (that is from data already held e.g. existing patients records) or prospectively (that is from data still to be collected) decide how many data will be collected i.e. the sample size define the start and finish dates design the data collection tool – this may take the form of a questionnaire, data collection sheet or checklist ethical issues must be considered; the data collected must relate only to the objectives of the audit, and staff and patient confidentiality must be respected – person identifiable information must not be collected Minimise or eliminate bias. Bias might arise from the sampling method, for example if you only issue questionnaires to patients who attend before 3pm you might not include any school children in your sample, or 9 if you include only staff who work in the practice on Mondays your sample might not be truly representative of the whole practice team. Stage 4. Plan to implement change. With the team, discuss the changes which are likely to be made to raise standard of care/ performance to the standard set. Plan how you will implement the change / s. It is more likely that this change will be effective if it is introduced to the whole team and they all agree to it. Change should be achievable and sustainable and might require some additional resource e.g. time / money or both. Stage 5. Apply for Clinical Audit Allowance approval and funding. Clinical Audit Allowance is only available for dentists in the General dental Services. Dentists in the General Dental Services have an obligation to complete 15 hours of audit or SEA in the three year period beginning 1st August 2010. This is a Terms of Service commitment with the Health Board with which they hold a list number. Clinical Audit Allowance is available to dentists only and dentists can claim for up to 15 hours in the three year period. Dentists wishing to apply for Clinical Audit Allowance should use form GP216A (rev 4/11) for audits and GP216B (rev 4/11) for SEA and apply to the regional NES postgraduate office for audit project approval. Both the GP216A and GP216 B are interactive forms and can be submitted directly to NES. Please see the sections “How to apply for Audit and SEA project approval” and “Paperwork and Claims Forms”. Please note all applications should be made by email. Verifiable CPD . Audit and SEA activities can also be counted as verifiable CPD for both dentists and DCPs if the criteria set by the GDC relating to verifiable CPD are met. If you require assistance to determine whether your project can count as verifiable CDP then contact your regional audit administrator who will assist you. Stage 6. Establish the baseline. When you have received approval for your audit (in the case of GDS dentists), collect baseline data about what is happening now in the area covered by your audit. Use an appropriate data collection tool. Stage 7: Compare performance with criteria and standards. Compare this baseline performance with the standard you have set and decide how you will implement change /s. Stage 8: Implementing change. This stage is likely to involve other team members and it may be necessary to hold a meeting of those involved and ensure everyone is clear about their role, the timelines involved and the next stage of data collection. Stage 9. Carry out a second round of data collection. At an appropriate time following the introduction of the change, carry out a second round of data collection. Use this data to assess whether the standards set have been achieved. 10 Stage 10. Analysing the data. This is the analysis stage, whereby the results of the data collection are compared with criteria and standards. The end stage of analysis is concluding how well the standards were met and, if applicable, identifying reasons why the standards were not met in all cases. In theory, any case where the standard (criteria or exceptions) was not met 100% suggests a potential for improvement. In practice, where standard results were close to 100%, it might be agreed that any further improvement will be impossible to obtain. This decision will depend on the topic area – in some ‘life or death’ type cases, it will be important to achieve 100%, in other areas a lower result might still be considered acceptable. Stage 11: Producing an audit report. Once the data from both rounds of data collection are analysed and conclusions drawn the report can be written. The report should contain the information required to demonstrate to the reviewers that all the steps have been undertaken and all the points considered e.g. sample size, bias and staff training / team meetings. The report should also present the data in a way that the reviewers can readily understand. Bar charts and graphs are useful for this and should be clearly labeled. All conclusions should be clear and a logical progression of the results. If the audit is for verifiable CPD purposes then there is still a need for the report to be clearly written and the results, conclusion, discussion and thoughts for further areas for audit identified. Remember the GDC may request that a copy of your audit be submitted as evidence of verifiable CPD. These audit reports will also be of value for Clinical Governance and Revalidation purposes. Please ensure all reports and associated paperwork are available as word or excel documents as they will be submitted to the audit administrator by email. NB audit projects and SEA should be completed within a 6 month period. This means that the final report should be submitted not later than 6 months after the approval for the audit / SEA was given. Stage 12: Clinical Audit Allowance. For dentists who wish to claim Clinical Audit Allowance, the report and associated documents should be emailed to the audit administrator at the regional NES postgraduate office. Copies of the relevant paperwork can be found in the section “Paperwork and Claims Forms”, please download them and use as required. You will be advised by email when your report has been approved, the number of audit hours awarded and the number of hours of Clinical Audit Allowance you can claim. You should keep a copy of your audit report in case it is needed for GCD CPD or revalidation purposes. The regional postgraduate office will not retain a copy of your audit / SEA once it has been signed off and returned to you. Stage 13: Re-audit: Sustaining Improvements. After an agreed period, it may be useful to repeat the audit. The same approach for identifying the sample, standards, methods and data analysis should be used to ensure comparability with the original audit. The re-audit should demonstrate that the changes have been implemented and that improvements have been made. If the first audit did not meet the agreed standard, changes may have been introduced and these can be re audited. The audit cycle then becomes an upward spiral of continuous improvement. This stage is critical to the successful outcome of an audit process - as it verifies whether the changes implemented have had an effect and to see if further improvements can be made. 11 Audit Planning Tool. Planning an Audit and Choosing an Audit Topic The following checklist will help you to ensure that you get the most value from your audit project Write down your provisional audit topic idea: Yes Consider which of the following apply to your audit idea: (Tick all applicable categories) No Is your audit a national priority area e.g. reducing caries incidence? Is you topic a priority identified by SEA? Is there clear evidence of "best practice" from literature e.g. SDCPE or SIGN guideline? Does the topic affect a large number of patients / staff? Does it involve a higher than usual risk to patients / staff? Have concerns about this aspect of care been raised i.e. do you suspect that there is a problem? Is there a potential to improve the effectiveness of patient care? Can cost effectiveness be improved? Has your topic idea changed? Finally, consider the following Remember - keep it short and simple Have you kept your audit question simple? Have you decided to investigate too many areas? Will the audit be easy carry out? Consider the time and resources it will require Do you have them? If not, can you get them or simplify the topic? Ability to effect change: Will it be easy to make changes to the way things are currently practised if your audit identifies a problem? Are you committed to making changes? How will you go about letting others know of the proposed changes? Who to involve Consider involving the following individuals: The dental team e.g. dentists, receptionists, nurses, hygienists, technicians and therapists. Experts in the clinical area you are going to audit Your local DCP / PDP / CPD tutors or audit facilitator Patients Are you auditing your own or another's practise? If the answer is "Someone else's practise", then firstly ask yourself “why” and if you are completely happy with the answer then make sure you get them on board from the outset. 12 Defining your aims and objectives. Please state the aim of your audit: Before you start your audit, you should be clear about what you are trying to do. Your aim should convey the overall purpose of your audit e.g. to improve the quality of radiographs. ___________________________________________________ ___________________________________________________ ___________________________________________________ ___________________________________________________ ___________________________________________________ ___________________________________________________ Please state the objectives of your audit: Your objectives should state what you specifically want to achieve by the end of your audit e.g. to improve the positioning of radiographs to improve the processing of films ___________________________________________________ ___________________________________________________ ___________________________________________________ ___________________________________________________ ___________________________________________________ ___________________________________________________ ___________________________________________________ Good objectives are SMART - Specific, Measurable, Achievable, Realistic and Timely and should help you fulfil your aim. 13 Defining your criteria and setting standards. The section on “Useful links and sources of gold standard evidence” will help you decide upon the criteria and standards to use. Consider the criteria you will use within your audit e.g. the statements which specify what is to be measured about the quality of care e.g. in bitewings, 2mm of tooth should be seen beyond the alveolar crest 1. 2. 3. 4. 5. 6. Please write down the criteria (ion) which you will be measuring. Your standards should state the proportion of times the criteria should be fulfilled e.g. in 80% of bitewings, 2mm of tooth should be seen beyond the alveolar crest. Standards can be drawn from the literature, or set by the audit team. They can be 'gold' standards e.g. set at 100% or reflect what the audit team believe to be the minimum acceptable level of patient care. 1. 2. 3. 4. 5. 6. Please write down the standard(s) for your audit NB - you should have a separate standard for each of your numbered criteria. Please write down how you identified these standards or where they are from e.g. SDCEP guidance on antibiotic prescribing 14 What data to collect. Inclusion criteria. 1. Have you defined the data you are trying to collect? Your inclusion criteria limit the scope of the data you will collect, e.g. patients with four or more standing posterior teeth 2. 3. 4. 5. 6. Please write the data you intend to collect for your audit. Will collecting this data allow you to meet your audit objectives? Are you collecting unnecessary data? Exclusion criteria 1. Have you considered the types of cases which you will exclude from the audit? e.g. patients with a severe gag-reflex 2. 3. 4. Please describe the data which will be excluded from your audit The types of data and where to obtain it. Retrospective / prospective data Your audit will be based on retrospective or prospective data Retrospective data is data that has already been collected, e.g. stored on patient record cards. Prospective data is new data that still needs to be collected, e.g. the number of prescriptions written over the next 2 months in line with SDCEP guidance. What your data source will be? 15 How much data to collect. What is the total number of potential patients / events that you could include in your audit? How long do you think it will take to collect this amount of data (e.g. weeks/months)? ______________________________________________ ________________________________________________ ___________________________________________________ _______________________________________________ Yes Do you have sufficient time to do this? No ___________________________________________________ _______________________________________________ If NO, what do you think would be a realistic sample size for your audit? How long do you think it will take to collect the data for this revised number of patients / events (e.g. x weeks/months) ________________________________________________ __________________________________________ Start date_______________________________________________ Finally, having considered all of the above, please indicate the proposed start/end date for your audit and the number of patients you anticipate will be included in the audit. End date_______________________________________________ Proposed no. of patients / events to be included____________________________ It is a good idea to limit the amount of data collected i.e. collect data for 50 cases or all cases over a 12 week period. Do not collect data indefinitely. 16 How to collect data. Bias It will be important to consider the way in which you collect your data in order that you do not introduce bias into your results. The following are common examples of bias: Only collecting data during quiet periods of the day Selectively choosing patients Collecting data during holiday periods Everyone involved should be aware of the potential for bias and be committed to consistent data collection. It may be preferable to reduce the time period for data collection to ensure consistent data collection and reduce bias. Please indicate the sampling method that you intend to follow, in order to minimise sampling bias, e.g. continuous data collection over two weeks, every 4th patient presenting with at least one untreated carious lesion, every complaint over the last 2 years etc. Is everyone involved in the data collection aware of your sampling plan? Data collection tools. In order to collect and store your data, you will need to design a data collection tool. There are three main types of data collection tools which can be used to gather information. ______________________________________________ ______________________________________________ ______________________________________________ ______________________________________________ ______________________________________________ Yes No ______________________________________________ ______________________________________________ Checklists Data collection sheets Questionnaires ______________________________________________ the ______________________________________________ If you need help with this please contact the audit administrator at the regional postgraduate office. ______________________________________________ Please write down method you will use. 17 Writing an Audit Report. An audit report follows the same principles as reports for other health or scientific purposes. It should contain sufficient information, data and reasoning to demonstrate that the writer has understood the principles of the audit process and has followed them to reach a valid conclusion. The information and data should be presented in a clear unambiguous manner with graphs, charts etc used to illustrate key points. The report need not be lengthy but should present the facts in a logical sequence. It is suggested that the report follows the heading below as this will help you present the information succinctly and in the correct order. If the audit planning tool was used, then the information entered at that stage will be the basis for the subsections of the audit report. The audit project will have been summarised on Annexe 1 of GP216 and this can be used as a covering top sheet for the report. Introduction The introduction will explain why the topic was chosen, who identified it and the relevance it has Include a statement which describe what you hoped to achieve by carrying out the audit and how it will lead to an improvement in patient care Inclusion criteria and standards Which was the relevant standard and why? What level was aimed at with regard to this standard eg 80% compliance or 100% compliance and why this level was appropriate List the aspects of patient care on which data was collected. Method In this section describe how the project was carried out. Include: The data collection method used The data collection tool (attach a copy) The sample size for each round of data collection - and how that was decided The time frame of the two rounds of data collection Clearly state exclusions from the data set Describe the source of the data e.g. retrospective, random sample, every 4th patient, patient records How bias was managed Who collected the data How was change implemented, who was involved Who else was involved Results Results should be expressed as simple tables, graphs or charts, which are clearly labelled and address the main aim of the project. Describe the results in the text. Keep conclusions for later. Any additional information which does not support the results should be presented as an appendix. Discussion State the findings clearly. Present changes made in a stand alone graph and explain the significance and how they were implemented. Reflect on what went well and what did not go quite so well. What might have been done differently to make more of and improvement. What further action is required. What are the wider implications of the results of this project. Acknowledgements Acknowledge help from all sources 18 References All references and reference material should be cited correctly in the text and listed properly at the end. All audit projects should be fully anonymised and contain no patient identifiable material. In addition no dentist, DCP or practice should be identifiable. Ideas for audit topics. As outlined in the introduction, it is likely that a SEA will alert you to the need to carry out an audit. However, the list below might give you a few more ideas, if you require a little more inspiration. If necessary contact your regional audit administrator who will direct you to assistance with the actual design of your audit. Use the SDCEP Practice Support Manual for audit ideas on the following (and more) Record keeping Storage of records Ethical Practise Confidentiality Data Protection Risk management Reporting for public safety Dealing with complaints Continuity planning Dealing with medical emergencies in practice Disability Discrimination Act Child Protection / Protection of Vulnerable Groups Dealing with Complaints Communication within the team and with other professionals Radiation protection o Radiation legislation and General Principles o Formal Appointments, Local Rules, Written Procedures o X ray equipment, Quality Assurance, Training o Radiation Protection File, Incidents And Accidents Health and Safety o H&S legislation, H&S Risk Management o Fire Safety, Electricity at Work o COSHH, Mercury Handling, Radiation Protection o Display Screen Equipment, Manual Handling o Infection Control, Waste Management, Personal Protective Equipment, Latex Allergy, Hand Hygiene o RIDDOR o Use of Pressure Vessels o Use of Lasers Prescribing drugs Using the SDCEP guidance “Drug Prescribing for Dentistry” to audit you recent prescribing An audit of the quality of clinical notes relating to LA s administered Radiography An audit to improve your compliance with different aspects of the IRMER 2000 regulations. An audit to improve the settings used for DPTs to minimise the exposure of head and neck tissues. A radiographic assessment of endodontic success. Effectiveness of secondary caries detection in class I and class II restorations using bitewing radiographs compared with clinical examinations. 19 Emergency Dental Care Use the SDCEP dental clinical guidance “Emergency Dental Care” to audit different aspects of your individual or your practices performance. Medical Histories An audit of the completeness of medical history records. Minimising Fail to Attends & Recall Appointments Audit of recall procedures, to improve success in recall attendance – for example, phone calls vs recall cards vs emails. An audit to develop a profile of patients likely to miss appointments. Crowns A comparison of the quality and cost effectiveness of crowns produced in our "in house "laboratory with those sent to outside labs. An audit of fibre post retained crowns. An audit of the durability of different types of temporary crowns and temporary restorations. Posterior Composites A practice wide audit of the information given and explanation of risk and longevity given to patients who request multi surface posterior composites. An audit of the longevity of posterior composites. As above in patients with differing standards of oral health. Dry Sockets & Socket Infection A prospective audit of dry socket prevalence following routine extractions. An analysis of the incidence of infected sockets after extraction: the problems related to the teeth beforehand and the procedures carried out when extracting the teeth. An audit of the incidence of dry sockets within a practice. Dentures An audit of the recall interval for complete denture wearers. An audit of the disinfection of the different stages of lab work – by the practice and the lab. Endodontics An audit of the 5-year success rate of molar endodontics, clinical and radiographic. A clinical audit of working length measurement during endodontic treatment. An audit of the costs associated with the use of rotary endodontic instruments. An audit of success of different techniques employed when using rotary instruments. Periodontal Screening A retrospective audit of recorded BPE screening and subsequent prescription of treatment. An audit of the quality of referrals from dentist to hygienist. An audit of periodontal notes in patients' record cards. Sealants A retrospective clinical audit into the longevity of fissure sealants placed in patients over a period of several years. An audit of the extent of sound retention of fissure sealant one year after being applied to teeth. 20 Paedodontics Use the SDCEP guidance “The Prevention and Management of Dental Caries in Children” to identify aspects to audit. An audit of how cases of dental trauma were handled in the practice. Illustration of possible audit topics, professional groups, sample size and CPD hours Audit Topic Relevant professional group What areas will be audited Sample size / time frame Medical History HT, DN PM, D DN Method of gathering information. Updated in line with Best Practice Maintenance of developer. Chemical changes. Process and mounting techniques. Equipment maintenance records of Availability of protocols. Staff safety awareness. Radiographic prescribing. Radiographic QA process. Measure accuracy of patient records. Updating systems. Storage. Raise awareness of patient waiting times for appointment. Implement better system. Awareness of waiting times of patients in practice 50 cases Possible number of CPD hours 5 hours 2 months 5 hours 2 months / 50 radiographs 5 hours 50 cases 5 hours 2 months 2.5 hours 2 months 2.5 hours Review current system for patient recall appointments and implement changes. Review current system for management and implement new protocols Monitor recording, maintenance and repair systems Monitor accuracy of information Recording systems of work to and from practice 12 months 5 hours 3 months 2.5 hours 3months 2.5 hours 2.5 hours Monitor the efficiency and traceability of lab work in and out of practice environment Measure compliance against SDCEP guidance and implement changes Ensure compliance of hand hygiene within environment Monitor efficiency of waste collection systems and protocols Measure efficiency of payment systems within practice and implement changes Monitor accuracy and efficiency of GP17 completion and submission Monitor stock rotation and availability Measure environment cleanliness 3 months 2.5 hours 3 months 2.5 hours 1 month 2.5 hours hours Radiograph Processing Compliance with IRMER. HT, DN PM, D Patient Records HT, DN PM, D Appointment Waiting Times PM, DN HT, D Length of time patients wait in waiting room. Recalls PM, DN HT, D Failed to attend PM, DN HT, D Equipment Care Laboratory Prescription (tickets / lab sheets) Laboratory transfer systems Disinfection of lab work DN, HT PM DT, D DN, PM Hand Hygiene DN, D DT, HT DN, HT DT, PM, D PM, DN, D Waste Disposal Systems to collect patient charges Processing of GP17 forms Stock control Surgery / Lab PM, DN HT, D DT, DN PM D, DN, DT PM, HT DN PM, DN, DT DN, HT 3months 3 months 3 months 2.5 hours 6 months 2.5 hours 2.5 hours 5 hours 6 months 3 months 21 environment maintenance Appointment times DT, D Oral Health Aids DN, HT PM DN, HT, D and implement changes to adhere to best practice guidelines Monitor time allocated for types of procedures e.g. toothaches and implement changes Monitor stock and promotion of OH aids in practice 2 months 2.5 hours 3 months 2.5 hours How to apply for SEA and Audit project approval Read the other sections of this pack, identify an audit topic or SEA. Use the audit planning tool and seek advice from the regional postgraduate centre if necessary.. DCPs Read the other sections of this pack, identify an audit topic or SEA. Use the audit planning tool and seek advice from the regional postgraduate centre if necessary. Audit and SEA are not “Terms of Service” requirements for DCPs and therefore do not actually require approval before they are carried out. However, if there are any concerns about the proposed audit project or SEA, then please follow the steps below and your audit / SEA proposal will be reviewed by the appropriate person at the regional postgraduate centre. If you require help with your Audit or SEA, fill in either the DCP Audit Project form or the SEA outline and SEA Report form Send these forms electronically to the regional audit administrator who will arrange assistance Once you have received the assistance required, carryout the audit and write the report. In the case of SEA, complete the report, if you have not already done so. If you would like feedback on the quality of your audit or SEA report, then send the completed report electronically to the audit administrator who will arrange for feedback to be given. Once you feel the reports are finalized, then you should retain them and note the number of verifiable CPD hours you feel appropriate in your CPD record (the illustration above can be used for guidance). Keep the report and the CPD record until 1 year after the end of your current CPD cycle. Please note, there may be regional variations to the steps above. Please confirm the process with your regional postgraduate office before you embark on your project. Dentists applying for audit project approval complete GP216A (rev 4/11) for audits and GP216B (rev 4/11) for SEAs before you carry out the project and send electronically to the regional audit administrator who will submit it for approval if more than one dentist is talking part in the audit, complete Annex 1 to GP216A (rev 4/11) or Annex 1 GP216B (rev 4/11) and send electronically to the regional audit administrator who will submit it for approval carry out the audit, write the report and submit electronically to the regional audit administrator who will submit it for assessment once approved, the regional audit administrator will send you Annex 2 of GP216A (rev 4/11) (the Clinical Audit Approval and Certification Form) or Annex 2 of GP216B (rev 4/11) (the SEA Approval and Certification form). This will certify the audit / SEA was acceptable and indicate the number of hours of audit activity undertaken and the number of hours of Clinical Audit Allowance you can claim. You can also claim verifiable CPD for the number of audit hours. Note the hours on your CPD record. Keep the audit / SEA report until 1 year after the end of your CPD cycle 22 Dentists applying for SEA approval Complete the Application for SEA Approval and SEA Report form and send it electronically, along with GP216B (rev 4/11) to the regional audit administrator who will submit it for approval Once approved the regional audit administrator will send you Annex 2 of GP216B (the Significant Event Analysis Approval and Certification Form). This will certify the audit was acceptable and indicate the number of hours of audit activity undertaken and the number of hours of Clinical Audit Allowance you can claim. You can also claim verifiable CPD for the number of audit hours. Note the hours on your CPD record. Keep the audit / SEA report until 1 year after the end of your CPD cycle 23 DCP Audit Project Form To be used if assistance is required. Send it electronically to the NES audit administrator with a copy of the audit planning tool. Part 1 – Applicant / Audit leader Details If this is a joint project, please provide details of other DCPs / Dentists participating in audit in the section below. Surname Forename Practice Address Postcode Tel Number Email Are other DCPS / Dentists if so, give details below (please expand this section as necessary) Surname Forename Practice Address Postcode Tel Number Email Part 2 – Project Details Title of Project: Short description of proposed project: What is the aim of your project? What is the proposed start date of your project? What criteria will be measured? What standard have you set? 24 What reference material have you used/ How many rounds of data do you intend to have to show change? How many rounds of data do you intend to have to show change? What will be the sample size of each round? How many people will be involved in your audit? Do you have the support of the practice owner or practice manager for this project? Will other practices be involved – please expand? Who else will be involved in this audit – dental nurses, therapists etc.? How many hours do you anticipate it will take to complete the audit and write it up? What will your involvement in the audit be? 1 Design and planning of audit 2 Attending educational meetings 3 Literature search for evidence of Best Practice 4 Data collection 5 Data analysis 6 Production of a report 7 Presentation of results 25 Application for SEA Approval & SEA Report Form For use by DCPs and dentists if assistance is required and by GDS dentists who wish to apply for NES approval for Clinical Audit Allowance. Please submit this electronically to the regional audit administrator. SEA Project Code Number Supplied by NHS Education for Scotland Part 1 – Applicant / s Details Please complete this section in BLOCK CAPITALS Where application is a joint project, please provide details of other DCPs / Dentists participating in audit in the section below. Surname ________________________________________ Forename ________________________________________ Practice Address ___________________________________ ___________________________________ Postcode _____________Tel Number __________________ Email ________________________________________ NHS list number if applicable _________________________ GDC Registration Number _________________________ Other DCPS / Dentists please expand this section as necessary Surname ________________________________________ Forename ________________________________________ Practice Address ___________________________________ ___________________________________ Postcode _____________ Tel Number __________________ 26 Email ________________________________________ GDC Registration Number __________________________________ Part 2 – SEA Details Give a brief summary of the event. This should include the date of the event, the dates and details of any meetings held and who the SEA lead was. Do not include any information that might identify a patient. Expand as required. _______________________________________________________ _______________________________________________________ How many SEA hours are you applying for? What happened? ____________ Feel free to expand these sections as required. 1. Describe what actually happened. 2. Describe the roles of all the individuals involved in the events. 3. Describe the setting where the event took place. 4. What was the impact, or potential impact, of the event? Why did it happen? 5. What were the underlying reasons (positive and negative) why the event happened as described? Consider the professionalism of the team, the presence or lack of a system, did a system fail, was there lack of knowledge or communication in the team? Reflection and learning 6. Look back and identify what the underlying issues were. What has happened since, how have things changed, how have people changed, how have roles changed? 7. Look at the perspectives of all involved. Was there a lack of training or knowledge? Did the team work well together? Analyse the lessons learned and the opportunities for change and improvement. 27 8. Who has been involved in the analysis of the event? What action has been taken? 9. Describe what has happened since the event and what action has been taken. What has changed? Has there been any training? Are there any new protocols in the practice? How was this done, who was involved, were there any barriers to these actions? Will the changes be monitored? 10. How useful has analysing this event been? development? How has it aided practice / personal 11. Has this SEA led on to a clinical audit? In the case of dentists this form should be accompanied by GP216B (rev 4/11) if Clinical Audit Allowance is being claimed. 28 How to claim Clinical Audit allowance Determination XI of the Statement of Dental Remuneration states that Clinical Audit Allowance is available for General Dental Practitioners. Up to 15 hours of Clinical Audit Allowance is available in a three-year period, with the duration of the first project being not less than 5 hours Audit projects and SEA must be competed and the report submitted to NES within six months of the audit / SEA being approved. Clinical Audit Allowance is claimed using forms GP216A and GP216B (rev 4/11) and GP217 (rev 11/10). Form GP216A (rev 4/11) is used to gain approval for audit projects. GP216B (rev 4/11) is to gain approval for SEA. Annex 2 (rev 4/11) of both GP216A and GP216B are used to certify completion of the projects. Form GP217 (rev 11/10) is used to trigger payment. Forms Application for SEA Approval & SEA Report Form should be used for Significant Event Analysis. If claiming Clinical Audit Allowance for SEA activity then use GP216B as detailed above. These are available in the section “Paperwork and Claims Forms” and are interactive. They can be completed on line and submitted directly to the NES audit administrator. Please do not submit paper copies. 29 Frequently Asked Questions If the answer to your question is not here please feel free to contact you regional audit administrator. Q. What is audit? Audit is the systematic critical analysis of the quality of care, including 1. The use of resources 2. The procedures used for diagnosis and treatment 3. The resulting outcome and quality of life for the patient " Audit is taking note of what we do, learning from it, and changing our practise if necessary." In a nutshell, audit is about taking a little time to examine in detail one aspect of our practise, working out ways how to improve it, implementing the changes and then evaluating the changes to see if they have actually made an improvement. Q. I am busy seeing patients / assisting the dentist / on the desk. Why should I bother to do an audit? Consider the following questions and responses:Am I willing to learn from my work and change when necessary? Participating in audit is a positive way of assessing your own performance honestly and objectively. You can also compare your results with peers. Am I interested in enhancing and updating my own knowledge and skills? Audit is a very powerful postgraduate learning tool as it allows groups of dental professionals to explore relevant topics in a cohesive and enthusiastic way, with much of their positive findings being implemented into their practise. Audit can also identify personal areas of strengths and weaknesses and help you to plan Continuing Professional Development activities. Are there any aspects of my practise which are annoying or frustrating? An audit will allow you to isolate, measure and analyse the problem and perhaps give you an insight how to solve it. This can be applied to any situation from losing record cards to evaluating failures of molar root treatments. Am I interested in improving the quality of care offered? All dental professionals want to provide their patients with a good quality of service. Audit can be used for areas where improvement is required and to demonstrate an improvement in patient care but it can also be used to reinforce just how well the staff are already doing. Audit used in the correct way can be used as a motivational and a team building tool. Q. Will an audit or SEA count towards Continuing Professional Development? 30 Yes, Clinical Audit and SEA can be used as verifiable and / or non-verifiable CPD hours if the manner in which is it carried out meets the criteria laid down by the General Dental Council. More details can be obtained from the GDC website at http://www.gdc-uk.org/ Assistance in producing audits / SEAs which meet the GDC criteria is available from the CPD tutors at the regional postgraduate centres. Dentists can claim Clinical Audit Allowance as described in Determination XI of the Statement of Dental Remuneration if the audit / SEA has been approved by the NES or a regional audit panel. Clinical audit fulfils an important part of clinical governance requirements for dental practitioners. It is recommended, therefore, that all practices, clinics or departments keep a record of all clinical audit activity. This can include in-house audit, dental team audit, collaborative audit, or VT audit. Q. What should I do an audit on? SEA may influence your choice of topic. Whatever audit you embark upon, it should relate to an area where you have a gut feeling that you are not providing the best service to your patients that you possibly can. This can cover any aspect of your practicing life from always running late, dealing with emergency patients, to the longevity of posterior composites. If a dental team is choosing a topic it is very important to reach a consensus to ensure all "stakeholders" are happy with the choice of topic. Choosing one individual’s pet subject can lead to poor compliance with the rest of the group. It is also very important to audit your own practice not someone else's, so make sure that at the outset of your project everyone who should be involved is involved in the correct capacity. Q. Can I get help with funding? Funding is available to General Dental Practitioners for participation in SEA or audit in the form of Clinical Audit Allowance. Clinical Audit allowance is awarded to projects by NES Regional Audit Assessment Panels. NES audit administrators can assist in the application for Clinical Audit Allowance. Contact details of the NES audit administrators can be found in the section “ NES regional audit administrators / audit contacts by region” GDPs are referred to Determination XI of the Statement of Dental remuneration, which details the number of hours and the rate of Clinical Audit Allowance. Funding for up to 15 hours Clinical Audit Allowance is available in a three year cycle and the initial audit in each cycle should be for not less that 5 hours. Audits and SEA should be completed within a six month period from the date on which they were approved. Since Clinical Audit activity is part of their Terms of Service for the Salaried Dental Service, there is no additional funding available for dentists in the Salaried Dental Service. Q. Where are the funding forms? The forms required can be found in “Paperwork and claims forms “ Clinical Audit Allowance is claimed using forms GP216A (rev 4/11) and GP217 (rev 11/10). Form GP216B (rev 4/11) is used to gain approval for audit and SEA projects and to certify completion of projects. Form GP217 (rev 11/10) is used to trigger payment. Q. If I submit my audit application form and GP216 electronically, how do I sign them? If you send your audit / SEA proposal electronically on Form GP216a or GP216B, (in the case of dentists) from your own email box, this will be an electronic signature for audit approval purposes. Q. Can I send my GP217 form electronically to PSD? No. PSD cannot handle electronic forms. You will have to send a paper copy of GP217 (rev 11/10) to PSD for payment. 31 NES regional audit administrators / audit contacts by region Please be aware, regional audit facilitators are available to support audit projects in some regions. Please contact the regional audit administrator for up to date contact and audit support details. West Regional Audit Administrator, Anne Bryden, anne.bryden@nes.scot .nhs.uk DCP Tutors, CPD Tutors, and West of Scotland Audit Assessment Panel, Clifton House, One Clifton Place , Glasgow. G3 7LD 0141 352 2800 North East Regional Audit Administrator, Fiona Rennie, fiona.rennie@nes.scot.nhs.uk DCP Tutors, CPD Tutors and North East of Scotland Audit Assessment Panel, Aberdeen Dental Education Centre, Argyll House Cornhill Road Foresterhill, Aberdeen, AB25 2ZR 01224 559708 South East Regional Audit Administrator, lynne.mcgillivray@nes.scot.nhs.uk DCP Tutors, CPD Tutors and South East of Scotland Audit Assessment Panel, Edinburgh Postgraduate Dental Institute 4th floor Lauriston Building Lauriston Place EDINBURGH EH3 9HA North Regional Audit Administrator, maryanne dinsdale@nes.scot.nhs.uk DCP Tutors, CPD Tutors and North of Scotland Audit Assessment Panel, Centre of Health Sciences, Raigmore Hospital, Inverness. IV2 3JH 01463255791 East Regional Audit Administrator, Joanne Elwin, joanne.elwin@nes.scot.nhs.uk Dundee Dental Education Centre, Frankland Building, Small’s Wynd, Dundee. DD1 4HN 01382 425771 32 This file was created by Oracle Reports. Please view this document in Page Layout mode. Useful links and sources of gold standard evidence Scottish Dental http://www.scottishdental.org This is an EXCELLENT site which links with hundreds of dentally related sites. Type your area of interest in the “search” box and see what appears. For example, type in SIGN guidance and you will find 20 relevant sites. General Dental Council www.gdc-uk.org British Dental Association www.bda.org Faculty of General Dental Practitioners www.fgdp.org.uk British Association of Dental Nurses www.badn.org.uk British Society for Dental Hygiene and Therapy www.bsdht.org.uk British Association of Dental Therapists www.badt.org.uk Dental Laboratories Association www.dla.org.uk Scottish Government www.scotland.gov.uk SIGN guidance www.sign.ac.uk Scottish Dental Clinical Effectiveness Programme (SDCEP) www.sdcep.org.uk Scottish Dental Practice Based Research Network www.sdpbrn.org.uk Childsmile www.child-smile.org Cochrane Oral Health Group www.ohg.cochrane.org Centre for Evidence Based Dentistry www.cebd.org 33 Direct Line: 34