3/10/2013
Developing an automated QC
program for ACR accreditation
compliance
*Kiaran P. McGee, Ph.D., Scott Stiving MS, Paul Bao
Ph.D. Dianna Lanners, Teresa Peterson, Renee
Jonsgaard
*Associate Professor of Radiologic Physics
Mayo Clinic & Foundation
©2012 MFMER | slide-1
Quality Control: Mayo Perspective
• Image quality a primary focus of Department
of Radiology mission.
• Quality control (QC) viewed as an essential
tool for maintaining optimal image quality of
imaging systems.
• In-house methods developed to monitor and
maintain image quality.
©2012 MFMER | slide-2
Mayo Clinic MR QC: Historical Review
• Technologists scanned manufacturer’s QC
phantom.
• Script run on scanner to extract and process
images
• Distortion and Signal-to-Noise (SNR) ratios
calculated and reported on file
• Program prompted user to rescan if failed or
proceed if passed.
• Two failures required call to QC technologists
©2012 MFMER | slide-3
1
3/10/2013
Manufacturer QC Phantom
Phantom and External Fiducials
Phantom
Fiducials
Phantom and RF TR
Head Coil
©2012 MFMER | slide-4
QC Data Files
©2012 MFMER | slide-5
QC Reports
©2012 MFMER | slide-6
2
3/10/2013
Manual QC: The Good, Bad & Ugly
 Stringent quality control
measures are preemptive:
able to identify problems
before equipment fails
Reduces downtime and
unscheduled maintenance
- Manual record keeping is
time consuming and labor
intensive.
©2012 MFMER | slide-7
Quality Control & Accreditation
Quality Control
• Quality control and
accreditation are
interlinked:
• Accrediting agencies
require establishment
of QC program for
maintenance of
accreditation
• QC programs provide
important, quantitative
feedback on system
performance.
Accreditation
©2012 MFMER | slide-8
What is Accreditation & Why Do It
• Definition of Accreditation: To recognize or
vouch for as conforming with a standard.1
• Why accredit imaging systems:
• Ensures minimum image quality for
diagnostic imaging
• QC initiatives provide method for ongoing
image quality assessment
• Allows comparison of image quality across
different institutions
• However, a prime motivator is financial.
1Merriam-Webster
dictionary
©2012 MFMER | slide-9
3
3/10/2013
Medicare Improvements for Patients
and Providers Act (MIPPA) of 2008
©2012 MFMER | slide-10
Advanced Diagnostic Imaging Services
MIPPA Act - 122 STAT. 2532, SEC 135.
Advanced Diagnostic Imaging Services:
I.
Diagnostic magnetic resonance imaging,
computed tomography, and nuclear
medicine (including positron emission
tomography).
II. ….. excluding X-ray, ultrasound, and
fluoroscopy.
©2012 MFMER | slide-11
UnitedHealthcare’s (UHc) new policy to reimburse
only for medical imaging performed in accredited
facilities. The new policy goes into effect on March
1, 2008.
©2012 MFMER | slide-12
4
3/10/2013
CMS Approved Accreditation Agencies
• The ABR is not the only agency approved by the
Centers for Medicare and Medicaid Services
(CMS) – the US federal agency which administers
Medicare, Medicaid and State Children’s Health
Insurance Programs.
• CMS recognizes three accreditation agencies:
• American College of Radiology (ACR)
• Intersocietal Accreditation Commission (IAC)
• The Joint Commission (TJC)
Ambulatory Care Accreditation Program
©2012 MFMER | slide-13
ACR MRI Accreditation Process
For sites to become ACR accredited they must:
1. Submit initial accreditation package
including phantom and clinical images.
2. Satisfy minimum image quality criteria for
those data.
3. Establish & maintain an ongoing weekly
quality control (QC) program in compliance
with ACR QC recommendations.
©2012 MFMER | slide-14
ACR MRI Weekly QC Standards
• Parameters derived from T1-weighted sagittal and axial images of ‘ACR
phantom’ :
Center frequency
Transmit Gain/Attenuation
Geometric accuracy
High-contrast spatial resolution
Slice thickness
Slice position
Image intensity uniformity
Percent-signal ghosting
Low-contrast object detectability
Signal-to-noise
• All phantom images are reviewed and free of artifacts
• MR Table and operator console are fully functional
• The following have been visually inspected:
Patient transport and magnet
Filming & Viewing
RF integrity
Facility safety
©2012 MFMER | slide-15
5
3/10/2013
ACR Recommended Testing Frequencies
Technologist’s Weekly QC Tests
Physicist / MR
Scientist Annual
QC Tests
• Center frequency
• Low-contrast resolution
• Magnetic field
homogeneity
• Table positioning
• Artifact analysis
• Slice position accuracy
• Setup and scanning
• Film quality control
• Slice thickness accuracy
• Geometric accuracy
• Visual checklist
• Radiofrequency coil
checks
• High contrast resolution
• Soft copy displays
©2012 MFMER | slide-16
Why Automate The QC Process?
• Mayo performs daily QC checks
• Compliance with ACR QC requirements was
taking ~ 30 minutes per MR scanner.
• Limited personnel resources to perform daily
QC.
• No incremental staffing to meet ACR QC
requirements.
©2012 MFMER | slide-17
Daily ACR QC Workload
Daily ACR QC – Single Scanner
Daily ACR QC – 8 Scanners
Other (450 min)
OtherDaily
(240 QC
min)(30 min)
ACR
ACR Daily QC (240 min)
In 2004, 8 MR scanners represented ~
30% of our total fleet (24 MR Scanners)
©2012 MFMER | slide-18
6
3/10/2013
The Bottom Line: Unsustainable
©2012 MFMER | slide-19
So you want to develop your own
automated QC program!
©2012 MFMER | slide-20
Our Experience
• Approximately 12 months of development by
team of two programmers for:
• Web and database development
• Algorithm development
• Testing
• Implementation
©2012 MFMER | slide-21
7
3/10/2013
Project Phases and Development
Define Project Scope & Deliverables
Web Page Design and Usability Testing
Algorithm Development & Testing
Clinical Rollout
0
1
2
3
4
5
6
7
8
9
10
11
12
Development Month
©2012 MFMER | slide-22
Specifications
• System must process ACR images from
multiple vendors (Siemens & GE)
• Process images of the ACR phantom data
acquired at multiple field strengths and RF
coil types.
• System must process data with minimum
delay time
• User friendly and accurate
• Display, report and analyze data
©2012 MFMER | slide-23
Web Application Overview
Institutional MR Scanners
Intranet Accessible User PC
Sagittal localizer &
axial T1-weighted
images of ACR QC
phantom DICOM
pushed to analysis
workstation
Daily QC review and
additional QC
checks performed
by technologists &
physicists through
web portal to QC
databases
Images processed to extract 7
ACR QC metrics
Daily ACR QC metrics
sent to SQL database
DICOM receiver and analysis server
Institutional SQL
Database
Institutional Web Server
Communication between
web server and database
allows interactive data
review and tolerance setting
©2012 MFMER | slide-24
8
3/10/2013
ACR QC: Initial Findings
• Experience is a great teacher:
• Largest cause of ACR QC failures were
not related to image quality but due to
poor phantom positioning.
• To address this problem, a immobilization
jig was built to ensure the phantom is
always positioned at the same place
within the head coil.
©2012 MFMER | slide-25
Phantom Positioning
Spirit Level will aid in positioning
with split head coils and Siemens
Head Matrix.
Poor positioning will decrease
SNR value and may result in a
repeat exam. Image is angled 5°.
©2012 MFMER | slide-26
©2012 MFMER | slide-27
9
3/10/2013
©2012 MFMER | slide-28
Head Coil Types
GE Split Head Coil
GE 8 Channel Head
Siemens Head Matrix
Siemens T/R Head
©2012 MFMER | slide-29
Mayo – ACR Daily QC: 5 Basic Steps
©2012 MFMER | slide-30
10
3/10/2013
Step 1: Prepare the Scanner the night
before
• Since the ACR requires a check of the
mechanical properties of the table, we
intentionally have the evening shift lower the
table at the end of the day.
• The next day the technologist must raise the
table to begin scanning thereby checking
the mechanical integrity of the table.
©2012 MFMER | slide-31
Park
position for
head coil
Snug fit in
platter
Positioning
the Phantom
Landmark at the
“Nose” crosshair.
NOT on the coil!
Raise the table every morning.
(Lower the table at the end of the
day.)
©2012 MFMER | slide-32
Step 2: Scan the ACR Phantom
Sagittal T1 Localizer
ACR Axial T1
Step 3: DICOM Push Images to Server
©2012 MFMER | slide-33
11
3/10/2013
ACR Profile Protocol
• Series 1: T1 Sagittal Scout - 1 image
• Series 2: T1 Axials - 11 images
Gamma Protocols includes 2 extra series:
• Series 3: T1 Hi Res Sag T1 - 5 images
• Series 4: T1 Hi Res Ax T1 - 5 image
©2012 MFMER | slide-34
Series 1: Image 1 Sagittal Scout
S
Control limits:
S-I distance
148 mm
+/- 2 mm
R
L
Center Offset
+/- 7mm
Averaging 4 distances
to test distortion.
Center of phantom
should match center
of image.
I
©2012 MFMER | slide-35
Prescribing Series 2 Axials
#11
#10
#8
#1
Midline
*Make sure the 1st slice is centered at
intersection of the bottom triangles!
©2012 MFMER | slide-36
12
3/10/2013
Step 3: Log onto the “ACR Profile QC”
website
©2012 MFMER | slide-37
Step 4: Review the Scanner Dashboard and
Select Your Scanner
Blue: Images
have not been
pushed to the
server.
Yellow:
Images
received;
awaiting tech
review
Green: Questions
answered and
image analysis
“Passed”
Red: Image
analysis “Failed”.
Rescan phantom
©2012 MFMER | slide-38
Step 5: Answer the ACR Questions
This person is responsible
for these questions.
©2012 MFMER | slide-39
13
3/10/2013
1.Confirm the scanner
2. Answer three questions
3. Enter any necessary comments
4. Click on “Update”
©2012 MFMER | slide-40
When Things Don’t Work Out
• Pass (Green ball)
Start your first patient!
• Fail (Red ball):
Reposition the ACR phantom
Check the head coil & platter
Repeat the ACR exam
©2012 MFMER | slide-41
Additional QC Features
• 3T
• Physics review page
• Gamma knife QC
©2012 MFMER | slide-42
14
3/10/2013
3T Imaging & ACR Specifications
• ACR phantom requirements very from 1.5T to
3.0T:
• Low Contrast Detectability (LCD):
1.5 T : Site Dependent*. ACR: failure if # of
spokes detected changes by > 3
3.0 T : Total number of spokes > 37
• Percent Image Uniformity (PIU):
• 1.5 T : > 90 %
• 3.0 T : > 82 %
*Medical Physicist determines recommended action levels
©2012 MFMER | slide-43
3T Imaging
• More and more imaging is being performed at 3T due to
theoretical doubling of SNR compared to 1.5T.
• ACR recognized this and provided 3T QC guidelines
1.5 T
3.0 T
©2012 MFMER | slide-44
Low Contrast Detectability:
1.4% Contrast Pattern
1.5 T: Transmit – Receive Coil
3.0 T: 8 Channel Receive Only
©2012 MFMER | slide-45
15
3/10/2013
Percent Image Uniformity
1.5 T: Transmit – Receive Coil
1.5 T : > 90 %
3.0 T: 8 Channel Receive Only
3.0 T : > 82 %
©2012 MFMER | slide-46
Physics Review Page
©2012 MFMER | slide-47
Graphical Review of Data
©2012 MFMER | slide-48
16
3/10/2013
Gamma Knife QC
? : No Data
X : Failed
: Passed
©2012 MFMER | slide-49
Gamma Knife QC
QC tolerance exceeded ( >1.0 mm in red )
Z = 190 mm
120 mm
Non shaded = volume of potential anatomical coverage extending
beyond the Leskell fiducials:
•Anatomy outside of Z = 40 to 160mm
•Anatomy outside of Y = 40 to 160mm
©2012 MFMER | slide-50
ACR Phantom
with fiducials
Rigid tubes filled with CuSO4 solution
©2012 MFMER | slide-51
17
3/10/2013
Axial & Sagittal Gamma Slices
©2012 MFMER | slide-52
ACR QC Gamma Knife Fiducials &
Imaging Slices
X anterior
Z ant
Y outside
left
Y
right
Z ant
inside
Z post
inside
Z post
outside
X posterior
©2012 MFMER | slide-53
g knife QC
First Scan
Option 1
Option 3
Option 2
Gamma Knife
Distortion OK
Stop
Gamma Knife Distortion OK but
Rad. Onc. Physics Need to Check
Gamma Knife FAILED, rescan
the ACR Gamma Phantom
Second Scan
Option 1
Gamma Knife
Distortion OK
Option 2
Option 3
Gamma Knife Distortion OK but
Rad. Onc. Physics Need to Check
NO GAMMA KNIFE
Imaging on this Magnet
©2012 MFMER | slide-54
18
3/10/2013
Outer volume
(white) +1.5
mm tolerance
Inner volume
(gray) +1 mm
tolerance
©2012 MFMER | slide-55
Distortion: Temporal Stability
©2012 MFMER | slide-56
Multi Center Usage
Studies Processed = 59775
Rochester, MN
35000
30000
25000
20000
15000
10000
5000
0
1.5T
3.0T
3.0T
1.5T
SMS
GEHC
Scottsdale, Az
Jacksonville, Fl
1200
1.5T
1000
3.0T
800
600
400
200
3.0T
0
1.5T
SMS
GEHC
7000
6000
5000
4000
3000
2000
1000
0
1.5T
3.0T
3.0T
1.5T
SMS
GEHC
©2012 MFMER | slide-57
19
3/10/2013
Artifacts: What can ACR failures tell
us about the performance of our
scanner
©2012 MFMER | slide-58
SNR and Equipment Replacement:
Gradient Amplifiers
Gradient Amplifier Replaced
©2012 MFMER | slide-59
Center Frequency: Shim Changes
Shim required every 6 months
Shim required once a year
©2012 MFMER | slide-60
20
3/10/2013
ACR QC Artifacts: Visual
Checklist Failure
1.5T Scanner, 8 channel receive only coil
SCIC post processing
1.5T Scanner 8 channel receive only coil
PURE post processing
©2012 MFMER | slide-61
Visual Checklist Failure: Vendor
Distortion Correction
©2012 MFMER | slide-62
Multiple Failures and Subtle Artifacts
Pre failure – top part of
image signal loss
Post failure – top part of
image signal loss corrected
©2012 MFMER | slide-63
21
3/10/2013
Interaction Between Two MR Scanners
©2012 MFMER | slide-64
ACR Metric Analysis for Inter Scanner
Comparison - PIU
Coils
A & B, Scanner
A
• Two
8-channel
‘identical’
head coils
• Percent image uniformity (PIU) values differ
between coils on same scanner
Coils A & B, Scanner B
• Data demonstrate:
• MR equipment not ‘identical’
• trouble shooting value of the PIU metric
©2012 MFMER | slide-65
Gamma Knife Distortion: Autoshim
Autoshim
AutoshimOFF
ON
©2012 MFMER | slide-66
22
3/10/2013
Receiver Coil Failure
• Example of PIU failure due
to a bad element in the 8
channel head coil.
• LCD tests passed.
10/19/2010
10/17/2010
10/18/2010
10/16/2010
10/14/2010
10/15/2010
10/12/2010
Date
10/13/2010
10/10/2010
10/11/2010
10/8/2010
10/9/2010
10/5/2010
10/6/2010
10/7/2010
10/2/2010
10/3/2010
10/4/2010
10/1/2010
Percent
Percent Image Uniformity
84
83
82
81
80
79
78
77
76
©2012 MFMER | slide-67
Environmental effects: Vibration
©2012 MFMER | slide-68
Environmental effects: Vibration
©2012 MFMER | slide-69
23
3/10/2013
Environmental effects: Vibration
BEFORE
AFTER
©2012 MFMER | slide-70
Vibration from scanner equipment: Cold Head
Cold Head OFF
Cold Head ON
Vibration induced
ghosting.
SNR and ghosting
metrics failure
©2012 MFMER | slide-71
Gradient Power Supply Failure
• Both ghosting and SNR metrics failed
• Effects were subtle enough not to induce distortions
©2012 MFMER | slide-72
24
3/10/2013
Gradient Power Supply Failure
Post Gradient Power Supply Replacement
©2012 MFMER | slide-73
RF Receive Chain Failures
• SNR dropped by 50%
• Multi coil enclosure assembly unit failure (RF receive
chain)
©2012 MFMER | slide-74
RF Receive Chain Failures
Post Multi coil Assembly Replacement
©2012 MFMER | slide-75
25
3/10/2013
Commercializing Your Idea
©2012 MFMER | slide-76
Commercializing Your Idea
• Make sure that you protect your intellectual
property (IP) - disclosures and patents.
• It is more challenging to achieve this outside
of larger academic environments.
• Remember that IP costs money!
• Start up businesses require significant
capital and expertise
• Unless you plan to do it yourself, you will
loose some control of your idea
• Conflict of interests
©2012 MFMER | slide-77
Protecting your IP
Working for a larger, academic institution
provides access to legal expertise necessary for
protecting your intellectual property.
©2012 MFMER | slide-78
26
3/10/2013
Commercializing the Mayo QC Tool
• Departmental initiative to develop ‘non
traditional’ revenue streams
• Identified investment group willing to partner
with institution
• Start up company formed to commercialize
technology
©2012 MFMER | slide-79
How OnPoint’s System Works
Institution
Institutional MR Scanners
1
OnPoint
Web Browser
Place Phantom
5
Technologists
Medical Physicist
Service Technicians
Chief Technologists
2
Internet
Image
Processing
OnPoint
QC
Servers
DICOM Push
4
Internet
OnPoint Exchange
Agent
Images
OnPoint
Cloud
3
Storage System
©2012 MFMER | slide-80
QC & Visual Check Tests
QC Test
Confidential Presentation
©2012 MFMER | slide-81
81
27
3/10/2013
QC & Visual Check Tests
Visual Check Test
©2012 MFMER | slide-82
82
Dashboards, Trends and Notifications:
QA Dashboard
©2012 MFMER | slide-83
Dashboards, Trends and Notifications:
Trends & Notifications
©2012 MFMER | slide-84
28
3/10/2013
Main Login Page
©2012 MFMER | slide-85
85
Manual Data Entry
©2012 MFMER | slide-86
86
Data Review Page
©2012 MFMER | slide-87
87
29
3/10/2013
Failure Notification
©2012 MFMER | slide-88
88
Report Summary Page
©2012 MFMER | slide-89
89
©2012 MFMER | slide-90
90
30
3/10/2013
Confidential Presentation
©2012 MFMER | slide-91
91
Conclusions
• Quality Control (QC) is an ever evolving
process.
• QC can be a time consuming.
• Automation is an effective method for
improving QC productivity.
• Automation requires significant initial and
ongoing investment of resources and time.
• Quantitative QC metrics provide useful and
sometimes specific information on equipment
performance.
©2012 MFMER | slide-92
Acknowledgements
Scott Stiving M.S.
Paul Bao Ph.D.
Dianna Lanners M.S.
Teresa Peterson B.S
Renee Jonsgaard B.S.
Brian Diaz (OnPoint Medical Solutions)
©2012 MFMER | slide-93
31