New England Chapter of the AAPM
2013 Winter Meeting
Newton, MA
February 1, 2013
Making High Tech Radiotherapy Safe
Howard Amols, Ph.D.
Memorial Sloan Kettering
Cancer Center
New York, USA
Radiation Therapy Accidents
Radiation Therapy safety problems
• WHO World Alliance for Patient Safety
– Literature review 1976-2007
– 3125 patients affected by errors in radiotherapy
– ~1% resulted in death (middle and high income countries*)
• 98% of errors occurred in planning stage or during introduction
of new systems or equipment
– Of errors without adverse effects
• 55% planning
• 25% new systems or equipment
• 9% information transfer
• 10% treatment delivery
– Mis-information or errors in data transfer- greatest bulk of “near
misses” in modern radiotherapy services
3 Myths
1.Radiation therapy accidents are rare
2.Most accidents happened long ago in the
developing world
3.You need new high technology to have an
accident
There’s been a focus on high-technology as the source
of RT accidents.
This is only partially true.
New technology is often a contributing factor, but
most RT disasters involving new technology required
significant help from humans in order to escalate.
Really big errors are a team effort!
(more than one person has to make a mistake)
A Major Accident Requires:
Equipment malfunction AND/OR an individual act
of negligence
AND
a deficient QA/QC program, bad
communications, inadequate training,
lack of teamwork
If commercial airlines had the same error rates as hospitals
there would be a major airline crash EVERY DAY.
There is something fundamentally wrong with the
historical method by which the medical profession has
approached QA/QC:
“Those who don't know history are
destined to repeat it.”
Edmund Burke
Radiation Accidents: Common Threads
new equipment + new software, but no new QA/QC
understaffed, overworked, undertrained, rushed
no internal redundancy, no external audits
no common sense, no time outs
bad communication, no central reporting
too much faith in manufacturer design
manufacturer and institutional denial
unusual clinical results ignored by physicians
A recent IAEA analysis of reported treatment errors
noted common threads:
1. Possibility of systematic software error never
considered;
2. Manufacturer failed to inform users of previous
incidents;
3. Error messages from system ignored or overridden;
5. Used software in unanticipated sequence that
confused the system;
6. Input wrong data, misunderstood input requirements;
7. No physical dose measurements made to confirm;
8. New or recently upgraded equipment;
Life is more complicated than it used to be!
• Control software for Varian Truebeam Linacr (for
example) has > 1 million lines of computer code!
• No single person can completely understand how
such a system works
• No government regulatory agency has the expertise
to review such a system
• Teamwork and communication is essential
RT is more complicated than it used to be!
• Manufacturers software engineers rarely have clinical
experience and do not know how a clinic really operates
• People will always find new ways to make mistakes that are
difficult to anticipated
• Quality Assurance must be an evolving process, to be
reviewed and modified whenever new technology is
implemented and when errors are discovered
• Study near-misses as well as actual errors
• Never underestimate the ingenuity of a fool, or someone
who is being rushed!
Special Dangers of Hi-Technology
1.Systematic errors harder to detect
2.Humans get complacent. Don’t really check
computers (evolution of R/V systems)
3.Many treatment components too complex for
humans to check (e.g., DMLC files, IMRT files)
4. Many treatment aids/devices are invisible
(dynamic wedge, MLC, etc.)
5. Errors made on day 1 can propagate
6. Programmers don’t understand what we do
7. We don’t understanding what programmers do
7. Too easy to `over ride’ warning messages
8. Manufacturers training programs often
inadequate
My `favorite’ IMRT accident
• Almost everything that can go wrong in a
radiation therapy accident occurred here!
• Best teaching tool I know of on the subject
of radiation therapy errors
• Background, IMRT accident
• March 2005, New York City
• A patient is to be treated with IMRT for head and
neck cancer (oropharynx)
• March 4 – 7:
An IMRT plan is prepared:
“1Oropharyn”. Verification plan created by TPS. EPID
dosimetry confirms correctness.
• March 8: patient treated correctly with “1Oropharyn”.
• March 9-11: Fractions #2, 3 and 4 also correct.
Verification images for the kV imaging system are
created and added to the plan, now called
“1AOropharyn”.
• March 11: Physician wants modified dose distribution
(reducing dose to teeth) “1AOropharyn” is copied and
saved to the DB as “1BOropharyn”
• NOTE the dates!
• March 14: Re-optimization for “1B Oropharyn”.
• New optimal fluences saved to DB.
• MLC motion control points for IMRT generated.
Normal completion.
What happened?
• March 14
• “Save all” is started. All new and modified data
should be saved to the DB.
• In this process, data is sent to a holding area on the
server, and not saved permanently until ALL data
elements have been received.
• Data to be saved included: (1) fluence data, (2) DRRs
and (3) MLC control points
What happened?
• March 14, 11 a.m.
• An error message is displayed.
• The user presses “Yes”, which begins a second,
separate, save transaction.
• MLC control point data is moved to the holding area.
The purpose of this error message is so that you can click `yes’
to proceed
What happened?
• March 14, 11.a.m.
• The DRR is, however, still locked into the faulty first
attempt to save.
• This means the second save won’t be able to
complete.
• The software would have appeared to be frozen.
Ctrl-Alt-Del usually gets you out of this so....
What happened?
• March 14, 11.a.m.
• Within 12 s, another workstation is used to open the
patients plan to load into VARIS and to treat.
• Unbeknownst to the users, the MLC motion file is
NOT properly saved because the error message was
over-ridden
What happened?
• March 14, 11 a.m.
• No verification plan, no pre-treatment dosimetry, no
review by 2nd physicist (not enough time!!)
• Several computer crashes ignored and over-ridden.
• Plan approved by physician (the plan looked OK on
printout, but the computer data files were corrupted)
Big Errors are usually a team effort!
So far:
• The radiation oncologist did not read port films on time, and
then rushed everybody else
• Neither therapists nor physicists confront him
• The treatment planner over-rode an error message they did not
understand
• The physicists did neither a double check, nor a dosimetry test of
an IMRT plan (also called `billing fraud’)
• There were real bugs in the software design (it crashed, two
people could open file at the same time)
• So far no real harm has been done!
• But they’re sowing the seeds for a really great accidents!
And now the therapists add the final piece!
March 14, 2005, 1 p.m.
• What they should have seen:
March 14, 2005, 1 p.m.: What they also didn’t notice:
1324 MU with MLC wide open
Discovery of accident
• March 14-16, 2005
• The patient is treated without MLCs for 3 fractions
• On March 16, a verification plan is created and run on
the treatment machine. The operator notices the
absence of MLCs.
• A second verification plan is created and run with the
same result.
• The patient plan is loaded and run, with the same result.
Impact of accident
• The patient received 13 Gy per fraction
for three fractions, i.e. 39 Gy in 3
fractions
Was there a bug in the Manufacuters software?
Yes, but the software bug needed a lot of help in order to
kill somebody:
1. MD didn’t check port films and rushed the plan
modification
1. Nobody confronted the MD
2. Error messages ignored and not investigated
3. Treatment plan QA not performed
4. Therapists weren’t watching MLC display
5. neither the manufacturer, nor the user, nor the FDA,
nor anyone else has the expertise to test this
software for safety
6. Accident not made public until much later!
Disasters are a team effort!
More radiation therapy accidents
Tampa, Fla., 2004-5. , 77 SRS patients overdosed >50%
because PDD factor not used in TG-51 calibration.
Uncovered after 1 year during RPC inspection for
participation in RTOG.
Springfield, Mo., 2004-09. 76 SRS patients overdosed >50
percent. Used too large a dosimeter to calibrate SRS
fields. No independent check, no mandated state or
federal reporting reguirement, no requirement for
physicists or therapists to be certified. Insufficient
support from manufacturer.
For 10mm cone
The Farmer chamber
is much larger than the
x-ray field, and most of
it is in the beam penumbra.
Dose reading =
(6+4x0.7)/24 = 37%
24mm
7mm
Penumbra
= 3-5 mm
Penumbra
= 3-5 mm
10mm
Note: dose uniformity should be +3% over 80% of
the field width (for 10 x 10 field size).
80% of field = 8mm. But for very small fields
uniformity is worse.
More radiation therapy accidents
Another radiosurgery error: Linac collimators not
properly set for small SRS cones (hospital staff assumed
this was done automatically)
SRS cone
Insert (2-3)
SRS cone
Holder
(5x5)
Linac
Collimators
(10x10)
More…..
UK, 1982-90: incorrect SSD correction (did not know how
TPS worked). 1045 patients, 30% underdose, >492 RT
failures
Bend, Oregon, 1980’s: incorrect TPC. 13% overdose
Spain, 1990: Linac `repair’ led to 36MeV e- beam no
matter what was programmed. No dosimetry check. 27
patients, 15 deaths
France,
2004: incorrect MU for dynamic wedge. 23
.
patients overdosed 20%, 4 deaths
Glasgow,2006: incorrect calculation of MU. Planner
thought TPS calculated MU/Gy and not MU/fraction. It
didn’t! 67% overdose results in death
France, 2006-7: large ion chamber used for SRS. 145
overdoses.
Errors we have Seen at Memorial
Sloan Kettering
MLC Error (FSRT) Event Sequence:





FSRS treatment scheduled right after DMLC treatment
At end of DMLC treatment leaves are all closed
BrainLab mMLC attached to Linac below regular MLC
Therapist does not retract primary MLC leaves
Primary MLC leaf position not detected by R&V (thinks it’s
SRS)
 Patient treated with correct mMLC apertures and closed
primary MLC
Why:
 Software not designed for two MLC’s
 Light field not normally used for SRS
DMLC Error: IMRT treatment with open MLC leaves:





DMLC field selected for treatment after a static MLC treatment
DMLC plan loaded, leaves retracted for light field use
“Go” selected, leaves fail to return to prescribed position
All systems allow treatment to proceed with retracted leaves
Therapist fails to detect error
Why:
 Software did anticipate this sequence of events
 Could only have happened with exactly the right wrong timing
 Very similar to Therac-25 disaster
Record/Verify Systematic Error with DMLC:






DMLC (v1) created from TPS and sent to R/V
Dosimetry checks done
Plan changed
MU (only) manually edited in R&V
No Independent Check of Data
Difference too subtle to see on Portal Image Check
Why:
 Human error
 Improper understanding of software (change in TP system
doesn’t automatically get transferred to R/V)
Upgraded,VARIS
to ARIA
Radiation Therapy Safety Program at MSKCC
• MSK was the first center in the US to initiate IMRT
• Since 1995 we have treated over 20,000 patients with IMRT
(> half a million treatment fractions)
• We never dismiss the potential for errors, and have put into place a
multi-tiered Quality Assurance Program to continually monitor all
phases of radiation therapy treatments at MSK
• Almost all radiation therapy accidents result from a combination of
equipment (or software) malfunctions PLUS human error
Radiation Therapy Safety Program at MSKCC
- training in all new technologies
- staff all board certified and licensed
- continuing staff education
- all errors, large and small are fully discussed at
monthly meetings of the Rad Onc QA Committee
- Minutes of Rad Onc QA Committee are reported
and discussed at the Hospitals QA Committee
- QA program is continually modified and upgraded
to reflect new findings and technologies
- New equipment and computer systems are tested
prior to release for patient treatments
The human component of most radiation therapy accidents results from
a combination of:
• failure to appreciate limitations of new technologies
• inadequate redundency in QA program:
- people checking people
- people checking computers
- computers checking people
- computers checking computers
• static or non-proactive QA program
Quality Assurance at MSK consists of
two equally important components :
• System wide QA of all treatment devices, hardware and
software
• Patient specific QA for each individual patients
customized
treatment plan
System wide QA of all treatment devices, hardware and software
• Annual dosimetry intercalibration test with Radiological Physics
Center
• Periodic dosimetry and treatment planning accreditation from
RTOG for IMRT and IGRT national protocol studies
• Periodic reviews by ACR (ASTRO??)
Using Linac Log Files
• Every day, during each treatment fraction for all patients all
treatment parameters (dose, mlc settings, x-ray energy, etc. are
recorded by the linac computer and stored in a `treatment log file’
• Every evening a `batch’ computer program is run comparing all
`IMRT treatment log files’ with the planned MLC files.
Discrepancies are investigated immediately by a medical physicist
• On a weekly basis all patient treatment folders are reviewed by both
a medical physicist and a radiation therapist to insure consistency
between delivered treatments and treatment plans
• On a weekly basis portal films and/or orthogonal x-rays are taken
on each patient to insure correctness of patient treatment position
Human errors usually fall into one of three categories:
1. Department policy is properly followed, but an error occurs
anyway. For example, policy requires that all treatment plans
and MU calculations be independently checked by a second
person before the patient's first treatment. This policy is
followed, but the second person also fails to detect the error;
2. Department policy is not followed. For example, no one
performs an independent check of treatment plan or dose
calculation prior to the first treatment; and
3. Department policy is deficient or incomplete. Most common
for new technology
4. Bizarre errors: sequence of events, almost impossible to foresee
or prevent
Nothing is foolproof for the sufficiently talented fool!
R/V systems, computer controlled Linacs, image guided
patient positioning systems, etc. reduce but do no prevent
errors. They enable humans to make different kinds of
mistakes faster and more efficiently.
New Paradigm for QA
Most errors are NOT systematic. They are patient
specific. Therefore QA should shift from equipment
focused to patient focused.
Patient Specific QA:
Treatment plan check (more difficult than before)
R/V, file check-sums (each fraction)
Independent MU check, dosimetry, portal images
Log file analysis, chamber measurement, film dosimetry
Machine Specific QA:
Film test
Dosimetry test
Drift test
MLC and IGRT tests
Is QA Reporting Like an Iceberg?
Reported
Errors
Discovered Errors
Actual number of errors
The Aviation Approach
The Hospital Approach
Even good people make
mistakes, they are the
the rule, not the exception
We are Gods. Mistakes are
the exceptions not the rule
Mistakes result from
flaws in the system
Mistakes result from
individual screw ups
Find the system flaw that
made it possible for a
particular individual
to make a mistake
Find and punish the person
who made the mistake
Start at the top and work
down the system until the
cause of the error is found
Start at the bottom and
work up. Find the lowest
ranking person you can
blame
They rarely make the same
Mistake twice
The Aviation Approach
The Hospital Approach
Even good people make
mistakes, they are the
the rule, not the exception
We are Gods. Mistakes are
the exceptions not the rule
Mistakes result from
flaws in the system
Mistakes result from
individual screw ups
Find the system flaw that
made it possible for a
particular individual
to make a mistake
Find and punish the person
who made the mistake
Start at the top and work
down the system until the
cause of the error is found
Start at the bottom and
work up. Find the lowest
ranking person you can
blame
They rarely make the same
Mistake twice
The Aviation Approach
The Hospital Approach
Even good people make
mistakes, they are the
the rule, not the exception
We are Gods. Mistakes are
the exceptions not the rule
Mistakes result from
flaws in the system
Mistakes result from
individual screw ups
Find the system flaw that
made it possible for a
particular individual
to make a mistake
Find and punish the person
who made the mistake
Start at the top and work
down the system until the
cause of the error is found
Start at the bottom and
work up. Find the lowest
ranking person you can
blame
They rarely make the same
Mistake twice
The Aviation Approach
The Hospital Approach
Even good people make
mistakes, they are the
the rule, not the exception
We are Gods. Mistakes are
the exceptions not the rule
Mistakes result from
flaws in the system
Mistakes result from
individual screw ups
Find the system flaw that
made it possible for a
particular individual
to make a mistake
Find and punish the person
who made the mistake
Start at the top and work
down the system until the
cause of the error is found
Start at the bottom and
work up. Find the lowest
ranking person you can
blame
They rarely make the same
Mistake twice
The Aviation Approach
The Hospital Approach
Even good people make
mistakes, they are the
the rule, not the exception
We are Gods. Mistakes are
the exceptions not the rule
Mistakes result from
flaws in the system
Mistakes result from
individual screw ups
Find the system flaw that
made it possible for a
particular individual
to make a mistake
Find and punish the person
who made the mistake
Start at the top and work
down the system until the
cause of the error is found
Start at the bottom and
work up. Find the lowest
ranking person you can
blame
They rarely make the same
Mistake twice
The Aviation Approach
The Hospital Approach
Even good people make
mistakes, they are the
the rule, not the exception
We are Gods. Mistakes are
the exceptions not the rule
Mistakes result from
flaws in the system
Mistakes result from
individual screw ups
Find the system flaw that
made it possible for a
particular individual
to make a mistake
Find and punish the person
who made the mistake
Start at the top and work
down the system until the
cause of the error is found
Start at the bottom and
work up. Find the lowest
ranking person you can
blame
They rarely make the same
Mistake twice
The Aviation Approach
The Hospital Approach
Even good people make
mistakes, they are the
the rule, not the exception
We are Gods. Mistakes are
the exceptions not the rule
Mistakes result from
flaws in the system
Mistakes result from
individual screw ups
Find the system flaw that
made it possible for a
particular individual
to make a mistake
Find and punish the person
who made the mistake
Start at the top and work
down the system until the
cause of the error is found
Start at the bottom and
work up. Find the lowest
ranking person you can
blame
They rarely make the same
Mistake twice
The Aviation Approach
The Hospital Approach
Even good people make
mistakes, they are the
the rule, not the exception
We are Gods. Mistakes are
the exceptions not the rule
Mistakes result from
flaws in the system
Mistakes result from
individual screw ups
Find the system flaw that
made it possible for a
particular individual
to make a mistake
Find and punish the person
who made the mistake
Start at the top and work
down the system until the
cause of the error is found
Start at the bottom and
work up. Find the lowest
ranking person you can
blame
They rarely make the same
mistake twice
Use of High Tech In Surgery
Position in
the pecking
order
MD’s
Nurses
Number
Of chances
to misuse
hi-tech
Technicians
e.g.; robotics, lasers, laproscopic
Use of Hi-Tech in RT: inverted training/culpability Pyramid
Position in
the pecking
order
Therapist
MD’s
Physicists
Dosimetrists
Dosimetrists
Physicists
Number
of chances
to misuse
hi-tech
MD’s
Therapists
e.g.; Linac, MLC, IGRT, R/V, treatment planning
Use of Hi-Tech in RT: inverted training/culpability Pyramid
Position in
the pecking
order
Therapist
MD’s
Physicists
Dosimetrists
Dosimetrists
Physicists
Number
of chances
to misuse
hi-tech
MD’s
Therapists
e.g.; Linac, MLC, IGRT, R/V, treatment planning