7/21/2014
Task-driven Imaging using
Advanced Reconstruction Methods
J. Webster Stayman
Johns Hopkins Biomedical Engineering
Acknowledgments
I-STAR Laboratory
Imaging for Surgery,
Therapy, and Radiology
www.jhu.edu/istar
Faculty and Scientists
G Gang
Y Otake
Clinicians
A Pourmorteza
G Gallia
J Prince
Z Gokaslan
J Siewerdsen
S Kawamoto
A Sisniega
A J Khanna
R Taylor
M Radvany
K Taguchi
D Reh
A Wang
M Sussman
W Zbijewski
Students
Q Cao
S Tilley
H Dang
A Uneri
S Reaungamornrat J Xu
S Ouadah
Funding
NIH KL2TR001077 (Ford)
NIH R21EB014964 (Stayman)
NIH R01CA127444 (Siewerdsen)
NIH R01CA112163 (Siewerdsen)
Varian Medical Systems
Siemens AX, Siemens XP
Advanced Reconstruction Methods
Tend to be implicitly defined optimizers of an objective function
ˆ  arg min (; y)
e.g., (  ; y)  F     y
( ; y)  A  F  y 
Enforce similarity between modeled projections of an object estimate and the data
Typically solved through iterative approximation
Relatively easy to generalize and include additional information
Various noise models – likelihood functions
(  ; y )   L   ; y 
General regularization strategies
Total variation, edge-preserving penalties
Constraints on reconstruction
Prior images and other prior knowledge
(  ; y )   L   ; y    R   
Indifferent to the specifics of the acquisition
Sampling – sparse acquisitions, redundancy, etc.
Geometry – helical vs. cone-beam vs. unusual geometries
1
7/21/2014
New Capabilities and New Choices
Fine control over regularization
Various kinds of regularization
Regularization strength
More exotic controls
Which one?
How strong?
Space-variant designs?
Capability for new data acquisition schemes
Fluence modulation
Sparse acquisitions
Arbitrary system geometries
How to dynamically change mA?
Which projections and how many?
What source-detector geometry is best?
The best answer depends on the task to be accomplished
What are you looking for?
Where are you looking for it?
How certain do you need to be?
Goal: Leverage capabilities associated with advanced model-based
approaches for optimized task performance.
Image Properties in Adv. Recon.
Image properties (e.g., noise and spatial resolution) are
Patient-dependent
Contrast-dependent
Position-dependent (nonstationary/space-variant)
-3
x 10
5
Object
-4
-3
x 10
8
Noise in Statistical Reconstruction
Noise in FBP Reconstruction
50
4
50
50
100
3
100
100
150
2
150
150
200
1
200
200
250
0
250
250
0
300
300
-2
350
350
6
-4
x 10
5
x 10
8
4
50
3
100
2
150
1
200
0
250
0
300
-2
6
4
-1
300
-2
350
-3
400
-4
450
400
-2
-4
-3
-6
450
300
400
-4
450
For prospective decision making, image property prediction is needed
200
-1
400
-5
100
200
300
400
100
-8
300
200
400
-5
2
350
-4
400
-6
450
100
200
300
400
-8
What kind of image quality measures can we use?
How do we contend with object-dependence?
Image Properties Prediction
Accurate predictions of image quality will require anatomical knowledge
Increasing availability of anatomical information prior to scanning
Longitudinal studies
disease progression
treatment assessments
Interventional imaging
intraoperative imaging, IGRT
Scout images in CT
3D scouts, PA/lateral scouts
Anatomical atlases (statistical atlases)
Low Exposure 3D Scouts (100 kVp, 6.8 mAs)
Care must be used in selecting appropriate image quality metrics
Many advanced reconstruction methods are highly space-variant/nonstationary
More sophisticated regularization (prior images, etc.) has added difficulty
Some methods exhibit locally space-invariant/stationary behavior permitting use of
Local noise power spectrum
Local modulation transfer function
-7
x 10
16
-0.8
-0.8
-0.6
-0.6 14
-0.4
-0.4 12
-0.2
-0.2 10
1
0.9
0.8
fy/mm
0.7
fy/mm
100
4
2
0
0.6
0 8
0.2
0.2 6
0.4
0.4 4
0.5
0.4
0.3
NPS
0.6
0.2
MTF
0.6 2
0.1
Consider advanced reconstruction methods with local stationarity
0.8
-0.8
-0.6
-0.4
-0.2
0
fx/mm
0.2
0.4
0.6
0.8
0.8
-0.8
-0.6
-0.4
-0.2
0
fx/mm
0.2
0.4
0.6
0.8
0
Penalized-likelihood reconstruction with (non-edge-preserving) quadratic penalty
2
7/21/2014
Penalized-Likelihood Reconstruction
y  Db exp  A 
L(  ; y)   yi log yi     yi   
i
ˆ  arg min (; y)  arg min   L(; y)   T R 
Analysis is potentially difficult due implicit definition and nonlinearity
but approximate expressions for local covariance and local point spread
function have been derived (Fessler, 1996)
Regularization-Dependence
Geometry-Dependence
Position-Dependence
Object-Dependence
cov{ˆ} j   AT D{y ( )}A+ R 
1
1
AT cov{y}A  AT D{y ( )}A+ R  e j
Regularizer
Strength
Location
Diagonal
j
weighting
Backprojector
Diagonal weighting
unit
vector
Projector + Backprojector
Projector Backprojector
Projector jth +
Regularizer
Strength Measurement
Covariance
1
PSFj   AT D{y ( )}A+ R  AT D{y ( )}Ae j
Location
j
Diagonal
weighting
Diagonal weighting
Backprojector
Regularizer
Strength
jth unit
vector
Projector + Backprojector
Projector
Fessler and Rogers, “Spatial resolution properties of penalized-likelihood image recon.: Space-invariant tomographs” Trans. Im. Proc. 5(9), 1996.
Stayman and Fessler, “Efficient calculation of resolution and covariance for penalized-likelihood recon. in fully 3-D SPECT,” Trans. Med. Im., 23 (12), 2004.
Noise & Resolution Prediction in PL
Geometry
Patient Anatomy
Location
Regularization
Strength
NPS
MTF
Predictor
Task
Performance
Local Noise Power Spectra
Theory
Empirical
1
1
2
3
2
3
Patient Anatomy
Local Modulation Transfer Functions
G. Gang, J. W. Stayman, W. Zbijewski, J. H. Siewerdsen, "Modeling and controlling nonstationary noise characteristics in filtered-backprojection and
penalized-likelihood image reconstruction," SPIE Medical Imaging, Orlando, FL, Vol. 8668, February 2013.
Task-Based Detectability Index
Detectability Index for a Non-Prewhitening Observer
Spatial Resolution
-7
x 10
16
-0.6
14
-0.4
12
-0.2
10
0
8
0.2
6
0.4
4
0.8
-0.8
-0.6
-0.4
-0.2
0
fx/mm
0.2
0.4
0.6
d j 
2
2
0.9
-0.6
0.8
-0.4
0.7
-0.2
 NPS j   MTFj WTask  df x df y df z
2
0.6
0
0.5
0.4
0.2
0.3
0.4
0.2
MTF
0.6
NPS
0.6
1
-0.8
  MTF  W 2 df df df 
j
Task
x
y
z
 

fy/mm
fy/mm
-0.8
2
0.8
Imaging Task Function
Noise
Imaging Task
0.8
-0.8
-0.6
-0.4
-0.2
0
fx/mm
0.2
0.4
0.6
0.1
0.8
0
Low Frequency Task High Frequency Task Directional Task
WTask  H1  H 2
H1: Stimulus present
H2: Stimulus not present
0.018
7
x 10-3
7
x 10-4
5
3
0.012
4
0.006
1
0
0
*ICRU54 “Medical imaging – the assessment of image quality”
3
7/21/2014
Task-Driven Regularization
Diagnostic Imaging
Reconstruct

Prior Knowledge of
Patient Anatomy
Predict/Optimize
Detectability Index
Task Definition

Task-Driven Regularization Design – Optimal Strength
Low Frequency Task
0.8
7
x 10-3
fy/mm-1
0.4
5
0
3
-0.4
1
-0.8
-0.8
-0.4
0
0.4
0.8
fx/mm-1
(1)  = 104.5
(2)  = 105.4
(3)  = 106.5
2
0.8
x10-6
0.4
fy/mm-1
(2)
2
(3)
Detectability index, d’
(1)
1
0
-0.4
1.6
NPS
-0.8
0
1
0.8
1.2
fy/mm-1
0.4
0.8
0
0.5
-0.4
104
105
106
Regularization strength, 
107
-0.8
-0.8
MTF
-0.4
0
0.4
0.8
-0.8
-0.4
fx/mm-1
0
0.4
0.8
-0.8
fx/mm-1
-0.4
0
0.4
0.8
0
fx/mm-1
Task-Driven Regularization – Multiple Locations
Low Frequency Task
0.8
7
x 10-3
fy/mm-1
0.4
5
0
3
-0.4
1
-0.8
-0.8
-0.4
0
0.4
0.8
fx/mm-1
Optimal  map, log10[ (x,y)]
2
6.0
Detectability index, d’
-80
5.8
1.6
-40
0.8
y/mm
5.6
0
1.2
5.4
* = 105.4
40
5.2
* = 105.1
104
80
105
106
Regularization strength, 
107
-100
-50
0
50
100
5.0
x/mm
4
7/21/2014
Task-Driven Regularization: Space-Variant Penalty
d‘ map, d’(x,y) from spatially varying  map
Low Frequency Task
0.8
Object
7
x 10-3
5
3
2.5
-40
0
3
y/mm
fy/mm-1
0.4
-80
-0.4
0
2
1
-0.8
-0.8
-0.4
0
0.4
40
0.8
1.5
fx/mm-1
80
-100
Optimal  map, log10[ (x,y)]
2.2
6.0
-50
0
x/mm
50
1
100
spatially varying  map
Global mean d’, <d’>
-80
5.8
-40
y/mm
5.6
0
5.4
40
5.2
80
-100
-50
0
50
2
1.8
Constant  map
1.6
5.0
100
x/mm
1.4 3
10
104
105
106
107
Regularization strength, 
108
Task-Driven Regularization: Space-Variant Penalty
d‘ map, d’(x,y) from spatially varying  map
High Frequency Task
0.8
Object
1
-80
7
x 10-4
-40
0.8
0
4
y/mm
fy/mm-1
0.4
-0.4
-0.8
-0.8
0
-0.4
0
0.4
0
0.6
40
0.8
fx/mm-1
80
0.4
-100
Optimal  map, log10[ (x,y)]
-50
0
x/mm
50
100
spatially varying  map
4.0
0.8
Global mean d’, <d’>
-80
3.9
0.6
-40
y/mm
3.8
0
0.4
3.7
40
3.6
Constant  map
0.2
80
-100
-50
0
50
3.5
100
x/mm
0
103
104
105
106
107
Regularization strength, 
108
Task-Based Regularization: Space-Variant Penalty
d‘ map, d’(x,y) from spatially varying  map
Asymmetric Task
0.8
0.018
fy/mm-1
0.4
0.012
6
-80
5
-40
0
0.006
-0.4
-0.8
-0.8
0
-0.4
0
0.4
y/mm
Object
0
4
40
0.8
3
fx/mm-1
80
-100
Optimal  map, log10[ (x,y)]
3.2
7
-50
0
x/mm
50
2
100
spatially varying  map
6.5
y/mm
-40
6
0
40
5.5
80
-100
-50
0
x/mm
50
100
Global mean d’, <d’>
-80
3
2.8
Constant  map
2.6
5
2.4
103
104
105
106
107
Regularization strength, 
108
5
7/21/2014
Task-Based Regularization: Space-Variant Penalty
d‘ map, d’(x,y) from spatially varying  map
Asymmetric Task
0.8
0.018
fy/mm-1
0.4
0.012
5
-80
4.5
-40
0
0.006
-0.4
-0.8
-0.8
0
-0.4
0
0.4
y/mm
Object
0
4
40
0.8
3.5
fx/mm-1
80
-100
Optimal  map, log10[ (x,y)]
-50
4.5
0
50
x/mm
3
100
spatially varying  map
7
6.5
y/mm
-40
6
0
40
5.5
80
-100
-50
0
50
100
Global mean d’, <d’>
-80
4
Constant  map
3.5
5
x/mm
3
103
104
105
106
107
108
Regularization strength, 
Task-Driven Geometry
Conventionally Ignored
by Interventional Devices
Diagnostic
Imaging
Preoperative
Image
Interventional Imaging
Planning
Data
Conventional
Intraoperative CT
Flat-Panel Detector
?
Task-Driven
Trajectory
Task
Definition
Traditional
Circular
Trajectory
Prior Information
about Patient and Task
X-ray
Source
Patient- and
Task-Driven
Intraoperative CT
Design Space and Model for
Robotic C-arms
Design Space
Model-Based Reconstruction
Forward Model
(qN,fN)
q
y     I 0 exp  A 
(q1,f1)
(q2,f2)
(q3,f3)
(q4,f4)
f
  q1 , f1  ,    Pq1 ,f1 


 
A
  

 q , f     P

  N N    q N ,fN 
Penalized-Likelihood Objective
ˆ  arg max  L  ; y    R   
Quadratic Regularization
R     12  T R
6
7/21/2014
Task-based Optimization of Geometry
Detectability Index (NPW Observer)
Spatial Resolution
d j 
2
  MTF  W 2 df df df 
j
Task
x
y
z
 

Predictors of Noise/Resolution for PLE
2
NPS j 
 NPS j   MTFj WTask  df x df y df z
2
Noise
MTFj 
Imaging Task

F AT D  y    Ae j


F AT D  y    Ae j   Re j


F AT D  y    Ae j

F AT D  y    Ae j   Re j

2

Optimization
 ˆ , ˆ   arg max d '  A  ,   ;W
 ,fˆ   arg max d '  A q ,f  ,
2
Task
,
qˆ
N 1
2
N 1
1
q ,f
1
,

, q N , fN  , q , f  ;WTask , 

Series of “Next Best View”
Optimizations
 q ,f  ;W
d '2 A
f
Task
,

-20
0
20
0
2
q1 , f1 
100
200
 q ,f  , q ,f  ;W
d' A
1
1
Task
q
,

   
0
q2 , f2  100
200
q
 
5
q3 , f3 
q30 , f30  q15 , f15 
q10 , f10 
-20
f 0
20
-20
f 0
20
 
AAq
A
;W
WTask
, ,,
ddd'd'22'2'2A
qq11q1,,f,f1f1,11f1,,, , ,,q,q2qq,2934241419f94 ,2f149341929424, q,,,qfq,,ff;W;Task
Task
0
qqq2552035,,f,ff5252035  100
200
10
15
q
20
25
35
30
Task-Optimizated Trajectory
-30
-20
-10
0
10
20
30
0
50
100
150
200
Task-Driven
215° Trajectory
Standard Circular
Short Scan Trajectory
7
7/21/2014
Reconstructions from Simulation Studies
Standard Circular Short Scan Trajectory
Task-Driven 215° Trajectory
J. W. Stayman and J. H. Siewerdsen, “Task-Based Trajectories in Iteratively Reconstructed Interventional Cone-Beam CT,” Int'l Mtg. Fully 3D Image
Recon. in Radiology and Nuc. Med., Lake Tahoe, June 16-21, 2013.
Testbench Investigations
Anthropomorphic Head Phantom
and Synthetic Vasculature
Modified CBCT Testbench
with Tilt Platform
Ability to step through entire embolization workflow
Initial CT for diagnosis and sizing of coils/stents
Intraoperative flouroscopy for coil embolization
Post-operative C-arm CT for assessment
Testbench Reconstructions
Preoperative Scan
360° Task-Driven Trajectory
360° Circular Scan
8
7/21/2014
Conclusions
Presented a general framework for task-driven imaging using advanced
reconstruction methods whereby one can optimize
*Regularization
*Acquisition geometry
Fluence modulation – automatic exposure control, fluence field modulation
Sparse acquisitions
Dose constraints
New paradigm for patient-specific and task-specific imaging
Customization of both acquisition and reconstruction
Lots of unanswered questions (aka Future Work)
Predictors for highly space-variant systems
Optimization using generalized task functions (beyond detectability)
How to quantify performance with prior image techniques (and other advanced methods)
9