IAEA Training Material on Radiation Protection in Radiotherapy Radiation Protection in Radiotherapy Part 12 Quality Assurance QUALITY as a goal “The totality of features or characteristics that bear on our ability to satisfy the stated or implied goal of effective patient care.” Comprehensive QA for Radiation Oncology, AAPM Task Group 40, 1994 To ensure the goal is reached requires a fully implemented Quality Assurance program throughout the facility. Radiation Protection in Radiotherapy Part 12: Quality assurance 2 What is Quality Assurance? “All those planned and systematic actions necessary to provide confidence that a product or service will satisfy given requirements for quality.” ISO 9000 Radiation Protection in Radiotherapy Part 12: Quality assurance 3 Quality Assurance In the BSS seen in the context of medical exposure as essential for radiation protection of the patient Quality Assurance and Control is also important to assess the overall effectiveness of protection and safety measures Radiation Protection in Radiotherapy Part 12: Quality assurance 4 Objectives To be familiar with the concepts of Quality Assurance as a multidisciplinary activity and its interrelation with radiation protection in radiotherapy. To be familiar with Quality Assurance procedures as a tool for reviewing and assessing the overall effectiveness of a radiation protection program. To be able to understand the need for and role of specific tests in the context of Quality Control and be able to identify appropriate national and international protocols for this task To be aware of the need for involvement of professionals in a Quality Assurance program and for radiation protection Radiation Protection in Radiotherapy Part 12: Quality assurance 5 Contents 1. Quality Assurance and the BSS 2. QA systems 3. Quality Control in radiotherapy External Beam RT Brachytherapy 4. QA and radiation protection Radiation Protection in Radiotherapy Part 12: Quality assurance 6 1. Quality Assurance and the BSS (Managerial Requirements) BSS 2.29. “Quality assurance programmes shall be established that provide, as appropriate: (a) adequate assurance that the specified requirements relating to protection and safety are satisfied; and (b) quality control mechanisms and procedures for reviewing and assessing the overall effectiveness of protection and safety measures.” Radiation Protection in Radiotherapy Part 12: Quality assurance 7 QA and QC Quality Assurance is the overall process which is supported by Quality Control activities Quality Control describes the actual mechanisms and procedures by which one can assure quality Radiation Protection in Radiotherapy Part 12: Quality assurance 8 Quality Assurance and Medical Exposure BSS appendix II.22. “Registrants and licensees, in addition to applying the relevant requirements for quality assurance specified elsewhere in the Standards, shall establish a comprehensive quality assurance programme for medical exposures with the participation of appropriate qualified experts in the relevant fields, such as radiophysics or radiopharmacy, taking into account the principles established by the WHO and the PAHO.” You must establish a QA program! Radiation Protection in Radiotherapy Part 12: Quality assurance 9 Relevant for radiotherapy WORLD HEALTH ORGANIZATION, Quality Assurance in Radiotherapy, WHO, Geneva (1988). PAN AMERICAN HEALTH ORGANIZATION, Publicación Cientifica No. 499, Control de Calidad en Radioterapia: Aspectos Clínicos y Físicos, PAHO, Washington, DC (1986). Radiation Protection in Radiotherapy Part 12: Quality assurance 10 BSS appendix II.23 “Quality assurance programmes for medical exposures shall include: (a) measurements of the physical parameters of the radiation generators, imaging devices and irradiation installations at the time of commissioning and periodically thereafter; (b) verification of the appropriate physical and clinical factors used in patient diagnosis or treatment; …” Check machine and data! Radiation Protection in Radiotherapy Part 12: Quality assurance 11 Consequences for radiotherapy A good acceptance testing and commissioning program is fundamental for any QA activities QA activities are typically a subset of the tests and procedures used for the commissioning of a unit QA applies to both physical and clinical aspects of the treatment Radiation Protection in Radiotherapy Part 12: Quality assurance 12 BSS appendix II.23 “Quality assurance programmes for medical exposures shall include: ... (c) written records of relevant procedures and results; (d) verification of the appropriate calibration and conditions of operation of dosimetry and monitoring equipment; .…” Radiation Protection in Radiotherapy Part 12: Quality assurance 13 Consequences for radiotherapy Treatment records must be kept of all relevant aspects of the treatment - including Session and Summary Record information Records all treatment parameters Dose Calculations Dose Measurements Particular emphasis is placed on QA of dosimetry Radiation Protection in Radiotherapy Part 12: Quality assurance 14 Treatment records Must contain all relevant information Can be in electronic format Radiation Protection in Radiotherapy Part 12: Quality assurance 15 BSS appendix II.23 “Quality assurance programmes for medical exposures shall include: ... and (e) as far as possible, regular and independent quality audit reviews of the quality assurance programme for radiotherapy procedures” Radiation Protection in Radiotherapy Part 12: Quality assurance 16 Consequences for radiotherapy A QA system itself and its outcomes must be critically reviewed External audits are recommended to verify that the checks are not only done but that they also achieve what they are supposed to do Every good system requires an independent look at times Radiation Protection in Radiotherapy Part 12: Quality assurance 17 2. QA systems Radiation Protection in Radiotherapy Many QA systems exist - one important example is the ISO 9000 system They are highly successful in manufacturing industry because they do improve productivity and avoid costly mistakes Part 12: Quality assurance 18 Good QA systems in radiotherapy Improves work practices Would have prevented most of the major accidents Radiation Protection in Radiotherapy Part 12: Quality assurance 19 ISO 9000 Comprehensive set of standards for QA (mainly in manufacturing and service industry) Adapted e.g. by ESTRO to the radiotherapy environment European Society for Therapeutic Radiology and Oncology (ESTRO) Advisory Report to the Commission of the European Union for the 'Europe Against Cancer Programme'. Quality Assurance in radiotherapy. Radiother. Oncol. 35: 61-73; 1995. Radiation Protection in Radiotherapy Part 12: Quality assurance 20 A Comprehensive Quality Assurance Program The details of such a program are often wrapped up in a “Code of Practice”. “Quality Assurance in Radiotherapy”, ESTRO Advisory Report, 1995 “Comprehensive QA for Radiation Oncology: Report of AAPM Radiation Therapy Committee Task Group 40”, 1994 “Quality Assurance in Radiotherapy”, WHO, 1988 Radiation Protection in Radiotherapy Part 12: Quality assurance 22 A Comprehensive QA Program typically comprises Quality Assurance Committee Policies and Procedures Manual Quality Assurance team Quality audit Resources Radiation Protection in Radiotherapy Part 12: Quality assurance 23 QA Committee Membership Must represent the many disciplines within the department Should be chaired by the Head of Department As a minimum must include a medical doctor, a physicist, a radiotherapy technologist and an engineer responsible for service and maintenance Must be appointed and supported by senior management Must have sufficient depth of experience to understand the implications of the process Must have the authority and access to the resources to instigate and carry out the QA process Radiation Protection in Radiotherapy Part 12: Quality assurance 24 Quality Assurance Committee Should ‘represent’ the department Should be ‘visible’ AND accessible to staff Oversees the entire Quality Assurance program Writes policies to ensure the quality of patient care Assists staff in tailoring the program to meet the needs of the Department (using published reports as a guide) Monitor and audit the program to ensure that each component is being performed and documented Radiation Protection in Radiotherapy Part 12: Quality assurance 25 Quality Assurance Committee Set agreed “Action Levels” Example: Physics is given the authority to ensure correct accelerator output For the daily output check two “Action Levels” are set For any daily measurement which exceeds 2% but less than 4%, treatment may continue but the Senior Physicist responsible must be notified (immediately) For any daily measurement which exceed 4%, treatment must STOP immediately and the problem investigated by the Senior Physicist responsible Radiation Protection in Radiotherapy Part 12: Quality assurance 26 Action levels Are quantitative Reflect the required outcome Are informed by the achievable outcome Must be unambiguous Should be easy to understand Radiation Protection in Radiotherapy Part 12: Quality assurance 27 QA Committee review Where “Action Levels” have been exceeded Where set procedures have been discovered to be faulty After a review, recommendations must be formulated in writing for improving the QA program When errors are discovered the fault often lies in the process rather than in the action of individuals Radiation Protection in Radiotherapy Part 12: Quality assurance 28 Documentation for the Quality Assurance Committee Terms of Reference The Committee must meet at established intervals and retain for audit purposes the minutes of its meetings, actions recommended and the results attained. In short, there is a QA program for the QA Committee Radiation Protection in Radiotherapy Part 12: Quality assurance 29 Policies and Procedures Manual This manual contains clear and concise statements of all the policies and procedures carried out in the Department Reviewed (typically) yearly Updated as procedures change Policies and Procedures Manual Radiation Protection in Radiotherapy Part 12: Quality assurance 30 Policies and Procedures Manual As a minimum, sections should exist for Administrative procedures Clinical procedures Treatment procedures Physics procedures Radiation safety Radiation Protection in Radiotherapy Part 12: Quality assurance 31 Policies and Procedures Manual It must be “signed off” by the Head of Department and appropriate section heads It is important that all staff have “ownership” to the manual - it should reflect the opinions of all and be agreed to by all A list of all copies of the Manual and their locations must be kept to ensure that each copy is updated Radiation Protection in Radiotherapy Part 12: Quality assurance 32 Quality Assurance Team Includes all disciplines Well defined responsibility and reporting structure Each member of the team must Know his/her responsibility Be trained to perform them Know what actions are to be taken should a test or action be outside the preset “action levels” Radiation Protection in Radiotherapy Part 12: Quality assurance 33 Responsibility Chart Leer Professional Area Radiation Protection in Radiotherapy Part 12: Quality assurance 34 Quality Assurance Team Each member of the team must also Have at least some understanding of the consequences when tests or actions are outside the “action levels” Maintain records documenting the frequency of performance, the results and the corrective action taken if necessary Radiation Protection in Radiotherapy Part 12: Quality assurance 35 Quality Audit “A systematic and independent examination and evaluation to determine whether quality activities and results comply with planned arrangements and whether the arrangements are implemented effectively and are suitable to achieve the objectives.” “Quality assurance in radiotherapy.”, Radiother. Oncol., 1995 Do you do what you say you do? Radiation Protection in Radiotherapy Part 12: Quality assurance 36 Quality Audit Ideally performed by someone outside of the organisation Examples IAEA/WHO TLD program for check of dose in therapy units EQUAL program in Europe Audits of clinical trials participation Radiation Protection in Radiotherapy Part 12: Quality assurance 37 Quality Assurance does not stop here! The Quality Assurance Committee and the Quality Assurance team must continuously monitor new information and implement this in their procedures Continuous Quality Improvement CQI - many other acronyms are available for this Part of virtually all QA systems Improved methods on cancer patient management are documented in clinical trial reports. Quality assurance protocols are continuously under development in many countries Regular Quality Assurance meeting for all members of a Section Continuing education - lectures, workshops, journal clubs and must be available for all staff Radiation Protection in Radiotherapy Part 12: Quality assurance 39 And finally: QA is not a threat, it is an opportunity It is essential in a QA program that all staff feel free to report errors A non threatening environment must exist Reward honesty with encouragement Education is the key, not punishment Radiation Protection in Radiotherapy Part 12: Quality assurance 40 3. Quality Control in radiotherapy Many documents exist that specify what QC activities should be performed in radiotherapy… Radiation Protection in Radiotherapy Part 12: Quality assurance 41 QC should ensure every step in the treatment chain... e.g.: hand calculation of treatment time Radiation Protection in Radiotherapy Part 12: Quality assurance e.g.: check source activity 42 Human errors in data transfer during the preparation and delivery of radiation treatment affecting the final result: "garbage in, garbage out" Leunens, G; Verstraete, J; Van den Bogaert, W; Van Dam, J; Dutreix, A; van der Schueren, E Department of Radiotherapy, University Hospital, St. Rafaël, Leuven, Belgium Abstract Due to the large number of steps and the number of persons involved in the preparation of a radiation treatment, the transfer of information from one step to the next is a very critical point. Errors due to inadequate transfer of information will be reflected in every next step and can seriously affect the final result of the treatment. We studied the frequency and the sources of the transfer errors. A total number of 464 new treatments has been checked over a period of 9 months (January to October 1990). Erroneous data transfer has been detected in 139/24,128 (less than 1%) of the transferred parameters; they affected 26% (119/464) of the checked treatments. Twenty-five of these deviations could have led to large geographical miss or important over- or underdosage (much more than 5%) of the organs in the irradiated volume, thus increasing the complications or decreasing the tumour control probability, if not corrected. Such major deviations, only occurring in 0.1% of the transferred parameters, affected 5% (25/464) of the new treatments. The sources of these large deviations were nearly always human mistakes, whereas a considerable number of the smaller deviations were, in fact, consciously taken decisions to deviate from the intended treatment. Nearly half of the major deviations were introduced during input of the data in the check-and-confirm system, demonstrating that a system aimed to prevent accidental errors, can lead to a considerable number of systematic errors if used as an uncontrolled set-up system. The results of this study show that human mistakes can seriously affect the outcome of patient treatments.(ABSTRACT TRUNCATED AT 250 WORDS) [Journal Article; In English; Netherlands] Radiother. Oncol. 1992: > 50 occasions of data transfer from one point to another for each patient! If one of them is wrong - the overall outcome is affected Radiation Protection in Radiotherapy Part 12: Quality assurance 43 QC activities in radiotherapy Three general areas: Physical dosimetry Treatment planning (dealt with part 10 lecture 3C of the course) Patient treatment Radiation Protection in Radiotherapy Part 12: Quality assurance 44 QC activities Must be planned prospectively daily weekly monthly annually whenever needed… The following is only a suggestion! Radiation Protection in Radiotherapy Part 12: Quality assurance 45 External Beam Radiotherapy Examples for daily QC Safety door and other interlocks radiation warning lights audiovisual radiation area monitor PTW Linacheck Radiation constancy check Mechanical/optical “pointers” Radiation Protection in Radiotherapy Part 12: Quality assurance 47 Test of optical components Used for patient set-up Essential Easy to perform Jigs available RMI test tool Radiation Protection in Radiotherapy Part 12: Quality assurance 48 Alignment of lasers for patient set-up Should point to the Gammex laser and test tool Radiation Protection in Radiotherapy isocentre Check also line width Check line alignment at least 20cm beyond isocentre Part 12: Quality assurance 49 Quality Control - Weekly Check of source positioning (cobalt 60) Couch movements (lateral, vertical, longitudinal) Radiation Protection in Radiotherapy Part 12: Quality assurance 50 Example for weekly QC summary From Constantinou 1992 Radiation Protection in Radiotherapy Part 12: Quality assurance 51 Quality Control - Monthly Dosimetry Output constancy Backup monitors Central axis %DD constancy Flatness/symmetry constancy Timer end effect Radiation Protection in Radiotherapy Part 12: Quality assurance 52 Quality Control - Monthly Safety interlocks emergency wedge etc Light/ radiation field coincidence Scales Isocentre position Cross hair position PTW Radiation Protection in Radiotherapy Part 12: Quality assurance 53 Test of optical components Quality Control - Monthly Used for patientRMI set-up Essential Easy to perform Field size indicators Jigs available Distance measuring indicators Jaw symmetry Latching of wedges, trays etc. test tool Wedge position (factorsRMI etc.) Radiation Protection in Radiotherapy Part 12: Quality assurance 54 Quality Control - Annual Dosimetry Safety Mechanical These checks are a scaled down version of the commissioning checks. It is a major QC exercise and is intended to validate the unit for another twelve months. Radiation Protection in Radiotherapy Part 12: Quality assurance 55 How to decide on frequency for tests? Likelihood of failure Severity of the consequences if something goes wrong Ease of the test - resources required This depends on local circumstances!!! Radiation Protection in Radiotherapy Part 12: Quality assurance 56 Time requirements for QC External beam per megavoltage unit daily: 30 minutes weekly: 2 hours monthly: > 4 hours annual: 2 days + Siemens Primus Linac These are estimates only - a qualified expert must decide on the actual requirements for a particular treatment unit Radiation Protection in Radiotherapy Part 12: Quality assurance 57 QC for Brachytherapy Sources The following QC should be done on receipt of the sources and documented Physical/chemical form Mentor Source encapsulation Radionuclide distribution and uniformity Autoradiograph Uniformity of activity amongst seeds Visual inspection of seeds in ribbons Radiation Protection in Radiotherapy Part 12: Quality assurance 58 QC for Brachytherapy Sources Calibration Do on receipt and document Ideal - every source Long half-life sources (e.g. Cs 137) Nucletron All Short half-life sources (e.g. I 125) If only a few, do them all If a large number, do a sample e.g. 10% Radiation Protection in Radiotherapy Part 12: Quality assurance 59 QC for Brachytherapy Sourcesmultiple seeds Suggested calibration tolerances Ideal mean of batch (3%) Deviation from mean (5%) Practical Review manufacturer’s documentation for tolerances Review ALL the manufacturer’s documentation Radiation Protection in Radiotherapy Part 12: Quality assurance 60 QC for Remote Afterloading Before each treatment day Room safety door interlocks Lights and alarms Radiation monitor Gammasonics Console functions Visual inspection of source guides Verify accuracy of ribbon preparation Radiation Protection in Radiotherapy Part 12: Quality assurance 61 QC for Remote Afterloading Weekly Accuracy of source and dummy loading Source positioning At each source change or quarterly Calibration Timer function Accuracy of source guides and connectors Radiation Protection in Radiotherapy Part 12: Quality assurance 62 QC for Remote Afterloading Annual Dose calculation algorithm Simulate emergency conditions Verify source inventory Radiation Protection in Radiotherapy Part 12: Quality assurance 63 QC Documentation Forms shall be established to guide the process easy to follow (even late in the evening after normal treatment has finished) diagrams useful Radiation Protection in Radiotherapy Part 12: Quality assurance 64 Examples for forms Radiation Protection in Radiotherapy Part 12: Quality assurance 65 Forms are useful for all tests Simple ticks may be sufficient Empty space for comments and drawings Radiation Protection in Radiotherapy Part 12: Quality assurance 66 Special equipment and procedures All equipment and all procedures should be tested To design a QC protocol, one needs to fully understand the goals and all steps of the procedure Requires a qualified expert Action levels should be set Radiation Protection in Radiotherapy Part 12: Quality assurance 67 A note on action levels Not too tight - one must be realistic about what can be achieved Not too lax - one must identify unsatisfactory practice As the practice improves, the action levels may be tightened Radiation Protection in Radiotherapy Part 12: Quality assurance 68 Not only treatment units require QC: Simulator Appropriate sections from the QC activities for a treatment unit kVp and mAs calibration Image intensifier quality checks Automatic exposure control if applicable Film processor Radiation Protection in Radiotherapy Part 12: Quality assurance 69 CT scanner Image quality Scaling and deformation Transfer of data Transfer of patient (is positioning OK, is couch on CT and linac identical?) Radiation Protection in Radiotherapy Gammex RMI CT test tool Part 12: Quality assurance 70 QC for Dosimetry Equipment Local standard 2 yearly calibration Field instruments yearly calibration PTW Linearity Leakage Recombination Radiation Protection in Radiotherapy Part 12: Quality assurance 71 QC for Measuring Equipment Automated scanners Positional accuracy Alignment Accuracy of data analysis Accessories Thermometer Barometer Radiation Protection in Radiotherapy Part 12: Quality assurance 72 Clinical QC Not only physics and dosimetry must be subject to QC, also clinical management A good way to do this are ‘chart rounds’ Radiation Protection in Radiotherapy Part 12: Quality assurance 73 Chart Rounds Regular review of patients Can be all patients or randomly selected patients Should include all patients with unexpected severe complications Radiation Protection in Radiotherapy Part 12: Quality assurance 74 4. QA and radiation protection Quality assurance is essential for a functioning system of radiation protection The BSS identifies the following areas in particular: Requirements for Practices Safety of Sources Medical Exposure Occupational Exposures Radiation Protection in Radiotherapy Part 12: Quality assurance 79 QA in medical exposures Physical QA as discussed before Process QA Leer Radiation Protection in Radiotherapy Part 12: Quality assurance 80 QA Program: Arrangements to be required from the licensee Procedures to establish patient identity Procedures to ensure accordance with prescription by a medical practitioner Procedures to ensure that radiation sources, including equipment can only be purchased from manufacturers and distributors authorized by the Regulatory Authority Radiation Protection in Radiotherapy Part 12: Quality assurance 81 Also the radiation protection program itself requires QA Check that the program meets its objectives Document improvements Document and rectify deficiencies Raise awareness Radiation Protection in Radiotherapy Part 12: Quality assurance 82 The cost of QA Dedicated staff - qualifications, training and numbers Equipment - include allowance for redundancy Time - commissioning, QA, reports, meetings, training Radiation Protection in Radiotherapy Part 12: Quality assurance 83 What do we get? Yes, correct - lots of documentation. But there are other benefits... Radiation Protection in Radiotherapy Part 12: Quality assurance 84 The benefits of QA Benefits for the department improved management system improved communication improved safety less duplication and waste Benefits to patients optimized procedure re-assurance Radiation Protection in Radiotherapy Part 12: Quality assurance 85 Additional benefits Credibility Potential to attract funding (and account for it…) Participation in multicenter clinical trials Regular updates and audits to continue the improvements Pride and confidence of staff Radiation Protection in Radiotherapy Part 12: Quality assurance 86 Involvement of Administration Without the support of the Administration the financial resources will not be made available The AAPM considers this to be so important that in their Quality Assurance policy they make the very first section “Part A: Information for Radiation Oncology Administrators”. “Educate those who control funding” Comprehensive QA for Radiation Oncology, Task Group 40, 1994 Radiation Protection in Radiotherapy Part 12: Quality assurance 87 But - Beware the Administration “Tick in the Box” syndrome! Administration will agree with QA They may even insist upon it Without education they will not understand what that really means in our environment Most administrators equate QA with an “audit” Many simply require the right boxes get ticked so they can be seen to do have done their job - this is not enough... Radiation Protection in Radiotherapy Part 12: Quality assurance 88 What do we risk without a Quality Assurance Program? Exeter, UK New cobalt 60 source installed Over the next 5 months, 207 patients were overdosed by 25% due to an incorrect calibration Contributing factors Calibration details not recorded Little documentation or protocols Radiation Protection in Radiotherapy Part 12: Quality assurance 89 What do we risk without a Quality Assurance Program? Exeter: Contributing factors (cont.) Reduced staffing levels (money) No independent check of calculations No independent check calibration It was detected during a Nation wide survey! Radiation Protection in Radiotherapy Part 12: Quality assurance 90 Where to get more information AAPM task group 40: Kutcher GJ, Coia L, Gillin M, Hanson W, Leibel S, Morton RJ, Palta J, Purdy J, Reinstein L, Svensson G, et al. Comprehensive QA for radiation oncology: report of AAPM therapy committee task group 40. Med Phys 1994;21:581-618. AAPM task group 53: Fraas, B. et al. Quality assurance for clinical radiotherapy treatment planning. Med. Phys. 25: 1773-1829; 1997. AAPM task group 56: Nath R.; Anderson L.; Meli J.; Olch A.; Stitt J. A.; Williamson J. Code of practice for brachytherapy physics: report of the AAPM Radiation Therapy Committee Task Group No 56. Med. Phys. 24:1558-98; 1997. ACPSEM Position Paper: Millar M, Cramb J, Das R, Ackerly T, Brown G, Webb D. ACPSEM Position Paper: Recommendations for the safe use of external beams and sealed brachytherapy sources in radiation oncology. Aust.Phys.Eng.Sci.Med. 1997; 20 (Supp): 1-35 Institute of Physical Sciences in Medicine. Commissioning and quality assurance of linear accelerators, IPSM report 54. York: IPSM; 1988. International Standards Organisation. Quality management and quality assurance standards. ISO 9000 series. PAN AMERICAN HEALTH ORGANIZATION, Publicación Cientifica No. 499, Control de Calidad en Radioterapia: Aspectos Clínicos y Físicos, PAHO, Washington, DC (1986). Radiation Protection in Radiotherapy Part 12: Quality assurance 91 WHO (World Health Organisation). Quality Assurance in radiotherapy. Geneva 1988. Radiation Protection in Radiotherapy Part 12: Quality assurance 92 Summary Quality Assurance is an essential part of radiotherapy It affects all aspects including the radiation protection program There are many different standards and guidelines for specific QA activities - it requires a qualified expert to choose the most appropriate for a particular center QA requires and encourages regular external audits QA is a continuous process - it is aimed at achieving improvements not laying blame. Radiation Protection in Radiotherapy Part 12: Quality assurance 93 Any questions? Question Please give an example for the concept of Continuous Quality Improvement from your practice. Example… just one of many A centre intends to improve treatment set-up. The measure patient positioning using port films on 20 patients e.g. of the prostate. The random variations are of the order of 8mm and the systematic error on average 9mm. The systematic error could at least partially attributed to different couch sag in simulator and treatment unit. This is reflected in update of the procedures. A repeat test shows that not only the systematic but also the random uncertainty have improved (the latter potentially because of heightened awareness). The smaller random variation allows to pick up other systematic errors… In addition to this the positive experience leads to the same tests to be done for other treatment sites... Radiation Protection in Radiotherapy Part 12: Quality assurance 96 Acknowledgment Lee Collins, Westmead Hospital, Sydney Lyn Oliver, Royal North Shore Hospital, Sydney Radiation Protection in Radiotherapy Part 12: Quality assurance 97