Towards Clinically-relevant
Standardization of Image Quality
• Ehsan Samei, Duke University
• Alan Rowberg, University of Washington
• Ellie Avraham, Eastman Kodak Company
• Craig Cornelius, Eastman Kodak Company
22-Sept-2003
1
Objectives
• Describe current medical image quality
and consistency performance efforts
• Identify limitations in existing standards
• Outline 3 specific proposals:
– Add new image quality factors to standards
– Update DICOM IQ performance services
– Research technical-clinical connection
22-Sept-2003
2
DICOM Image Consistency Efforts
• Three DICOM initiatives (1998-99):
– Grayscale Standard Display Function (GSDF)
– Presentation LUT (P-LUT)
– Grayscale Softcopy Presentation State (GSPS)
• And in actual practice…
– Are widely & effectively implemented
– Via IHE Consistent Presentation of Images
solution:
• Promoted conformance testing
• Demonstrated inter-vendor consistency
22-Sept-2003
3
DICOM GSDF: Barten Curve
1000
~150 cd/m2
L2
100
Absolute
luminance
increment
L1 << L2
10
L1
1
~3 cd/m2
0.1
0
200
400
J1
600
J2
Number of perceptual levels J1 = J2
22-Sept-2003
4
AAPM Task Group 18 Efforts
• Team of academic, clinical, and industry
contributors
• “Assessment of Display Performance for
Medical Imaging Systems” (2002):
– Practical guidelines for qualitative & quantitative
display assessment
– Includes all key aspects of display performance
– Defines IQ test patterns and procedures
– Recommends specific IQ acceptance criteria
22-Sept-2003
5
Standardization gaps
• DICOM GSDF:
– Pros:
• Mathematical definition
based on Human Visual
System model
– Limitations:
• Tonescale consistency
only: no other IQ factors
• No acceptance criteria /
conformance procedures
• Only for grayscale images
• No Display Device
Services (Capabilities)
22-Sept-2003
• AAPM TG18 report:
– Pros:
• Provides Professional
recommendations
• Covers all key display
performance aspects
• Acceptance Criteria
• Quantitative measures
– Limitations:
• Only guidelines
• Not a “standard”
6
The technical-clinical gap:
• The connection between quantifiable IQ
metrics & clinical performance is unknown:
– Luminance: deviations from GSDF, number of
gray levels displayed
– Spatial: resolution, noise, geometric distortion
– Chromaticity variations
– Environmental: ambient light, glare, reflection,
–…
• Amount of acceptable variation is unknown
22-Sept-2003
7
3 proposals to bridge the gaps:
1. Extend DICOM standard beyond
luminance response
2. Add and update DICOM Service
Classes for image quality /
performance
3. Promote research on clinical technical image quality relationship
22-Sept-2003
8
1. Display Image Quality (DIQ) Initiative
• Add measurable and quantifiable elements of
AAPM display performance procedures
• Include testing methodologies & defined limits
for clinical / diagnostic performance
• Quantify visual performance using
– Simple test images
– Specific observer protocols
– Relative acceptance indicators
• For both softcopy and hardcopy presentation
22-Sept-2003
9
DIQ Softcopy Examples
•
•
•
•
Quantify % deviation from GSDF curve
Define criteria for min & max luminance
Define visual luminance evaluation
Evaluate specific image quality factors:
– Ambient light limits: specular and diffuse
– Spatial resolution with TG18-QC/CX
– Check geometric distortion with TG18-QC
22-Sept-2003
10
Contrast response comparison
0.100
dL/L for a JND
GSDF contrast
+/- 10%
237
cd/m2
0.010
1.1
cd/m2
Non-standardized
display contrast
0.001
0
100
200
300
400
500
600
700
JND index
22-Sept-2003
11
AAPM TG18 Patterns
TG18-CT:contrast /
luminance response
22-Sept-2003
TG18-MP: bit-depth /
continuous grayscale
12
Comprehensive TG18-QC
The comprehensive
TG18-QC test pattern
for evaluation of key
display characteristics:
• Resolution
•Luminance
•Geometric distortion
22-Sept-2003
13
Other DIQ Extensions
• Hardcopy quality metrics:
– Media & printer quality (e.g., visible coating
variations, distortions, artifacts)
– GSDF compliance, # of JNDs theory vs. actual
– Printable matrix size
– Spatial frequency response fidelity
• Color extensions:
– Standardize for grayscale areas of color images
– Add descriptions for color image characterization
– Color display and print device calibration
22-Sept-2003
14
2. DICOM Service Extensions
• Add Display Performance Service Class
– Query image quality / performance information
– Control / configure manageable settings
– Include new IQ factors, e.g., MTF, ambient, …
• Extend Printer Configuration Retrieval Service
Class
– Include access to additional IQ factors
Note: Measurables include both human-evaluated and
automatically-measured values
22-Sept-2003
15
Use cases: Standardizing Output
• Printing application
– Retrieves matrix size and MTF of film printer
– Determines type of magnification, if any, to be
applied to the image for smallest artifact
• Display Performance Service Class User
– Requests the luminance characteristic curve from
its workstation’s display system
– Determines if the display is standardized
– If needed, computes an internal image tonescale
correction, producing GSDF standardized result
22-Sept-2003
16
Real device  standardized performance
300.00
250.00
Target GSDF
200.00
Luminance
Actual device
150.00
100.00
Desired
output
Modified Input
345
50.00
Original Input
443
0.00
0
100
200
300
400
500
600
700
Digital Value
22-Sept-2003
17
Use Case: Quality Control Management
• Centralized management application:
– Queries devices for calibration date, luminance
characteristics, ambient light settings, etc.
– Records the results in a central database
– Creates maintenance lists for displays and printers
• Reporting application uses database for:
– Regulatory and management reports
– Stability and lifetime statistics on displays to
support replacement schedules and budgets
22-Sept-2003
18
Network-wide Quality Control
Diagnostic display
Summary
data
Administrator’s
Characteristic curve, MTF,
other performance measures
Web-based
displays
22-Sept-2003
Request
Clinical display
Response
Modality Capture
console
Film
Printer
Characteristic curve, MTF,
other performance measures
Reflective Hardcopy
19
More use cases…
• Consultation:
– Maximize perceptual similarity to ensure “What I
see is what you will get!”
– Display performance information gives confidence
• Capture Consoles:
– “The technologist sees what the doctor will get.”
– Reduce errors, retakes, miscommunication
– Extend quality control program to consoles
22-Sept-2003
20
3. Clinical Significant of IQ Measures
We assume
there is a
connection…
GSDF vs. Monitor gamma differences
120
100
Luminance cd/m^2
80
Hmm…
60
40
20
0
0
50
100
150
200
250
300
-20
digital value
Physical metrics
22-Sept-2003
What deviations
matter, and how
much?
Clinical performance
22
Clinical impact: unknown
Rendered with DICOM GSDF
22-Sept-2003
Not rendered with DICOM GSDF
23
Goals of proposed research:
• Determine the clinical consequence of
variations in image quality metrics (e.g.,
GSDF conformance, MTF, noise, …)
• Define what constitutes image quality from a
diagnostic perspective
• Incorporate results into new joint standards
that will utilize standardized test patterns,
procedures, and clinical use cases
22-Sept-2003
24
Suggested research approach
• Form inter-society
committee to:
1. Design specific research
projects
2. Obtains & review data sets
3. Solicit & encourage active
participation by researchers:
radiologists, scientists, …
4. Arrange reporting of results
5. Define recommendations to
standards and professional
groups
22-Sept-2003
• Sample research:
1. Obtain images from 3
radiographic modalities
2. Present images,
simulating nonstandard
display behavior
3. Run observer
performance experiments
at major professional
meetings and events
4. Analyze by ROC methods
25
Conclusions
• Existing standards are insufficient to
assure consistent, high quality medical
image output.
• Steps are proposed to further the reach
and impact of DICOM toward quality
medicine.
• New directions will provide benefits for
PACS users, administrators, vendors,
and patients.
22-Sept-2003
26
Contact Information
•
Ehsan Samei, Duke University
samei@duke.edu
•
Alan Rowberg, M.D., University of Washington,
arowberg@earthlink.com
•
Ellie Avraham, Eastman Kodak Company,
ellie.avraham@kodak.com
•
Craig Cornelius, Eastman Kodak Company,
craig.cornelius@kodak.com
•
AAPM Task Group 18 web site:
http://deckard.duhs.duke.edu/tg18
22-Sept-2003
27