NA-MIC
National Alliance for Medical Image Computing
http://na-mic.org
Large-Scale Computing
Frameworks for
Developing Image
Analysis Tools
Steve Pieper, PhD
Isomics, Inc.
Founder and CEO of Isomics, Inc. a technology development company that works closely with multiple research institutions on projects
including NIH grants and software development. Much of this work is done in collaboration with the Surgical Planning Laboratory at BWH.
Acknowledgments
•
F. Jolesz, R. Kikinis, C. Tempany, P. Black, S. Wells, CF. Westin, M.
Halle, N. Hata, T. Kapur, A.Tannenbaum, M. Shenton, E. Grimson,
P.Golland, W.Schroeder, J. Miller, N. Aucoin, K. Hayes, A.
Yarmarkovich, C. Lisle, D. Marcus, J. Miller, R. Gollub, S. Pujol, S.
Barre, W. Plesniak, B. Fischl, D. Greve and many more….
V E
2
National Alliance for Medical Image Computing
http://na-mic.org
R I
TAS
Overview
• Software Infrastructure
– NA-MIC Kit Components
– 3D Slicer
• Modules and Extensibility
• Population Analysis
• Organization and Community
3
National Alliance for Medical Image Computing
http://na-mic.org
Large Scale Computing
• Working at a High Level
– Major Functional Components are Readily Available
– Tested, Cross Platform, Documented…
• Suite of Tools and Systems for Organizing and Processing
– Databases and Batch Computing
• Mechanisms to Translate Research to Clinical Users
– Plug New Research into a Usable Platform
• A Talented and Productive Community
– Multidisciplinary
– National and International
4
National Alliance for Medical Image Computing
http://na-mic.org
NA-MIC Kit
• The NA-MIC Kit is a Collection of Software and
Methodologies for Medical Image Computing
• Most components of the NA-MIC kit predate NAMIC (National Alliance for Medical Image
Computing)
5
National Alliance for Medical Image Computing
http://na-mic.org
NA-MIC Kit
• Packaging of:
– Applications
– Algorithms (toolkits)
– Methodologies
6
National Alliance for Medical Image Computing
http://na-mic.org
NA-MIC Kit Components
• End User Application
– 3D Slicer
• Image Analysis, Visualization, and
GUI libraries
– ITK, VTK, KWWidgets, Teem
• Archive and Distributed Processing Tools
– XNAT, Batchmake, BIRN GRID tools
• Software Engineering Tools
– CMake, CDash, CTest, CPack
http://www.na-mic.org/Wiki/index.php/SoftwareInventory
7
National Alliance for Medical Image Computing
http://na-mic.org
Provided by Pieper, Kikinis
Visualization Toolkit - vtk
Open source toolkit for scientific visualization,
computer graphics, and image processing
8
National Alliance for Medical Image Computing
http://na-mic.org
With support from several
IC’s
9
National Alliance for Medical Image Computing
http://na-mic.org
KWWidgets
•
•
•
•
Usage
– Cross-platform GUI with open source license
– Object Oriented C++ API (very VTK-like)
Active Development
– New widgets
– Work flow support
– File/directory browser
Future
– Tracing (for testing)
– Registration inspection widget
– Camera control widget
– Interface to ITK (spatial objects)
– Cleanup
kwwidgets.org
– cvs, dashboard, bug tracker, wiki…
– BSD license
10
National Alliance for Medical Image Computing
http://na-mic.org
NA-MIC Software Process
Version Control
Svn, CVS with
web access
Multiplatform Software
Compilation, Testing
CDash
Testing Dashboard
CMake, CTest
Source
Developers
check-in code
11
National Alliance for Medical Image Computing
http://na-mic.org
Developers
review results
Development Methodology
Extreme lifecycle
Release
Patch
Nightly
Continuous
Testing
Private
Sandbox
Release X.Y.1
NA-MIC
Sandbox
Release X.Y.2
Slicer
2 months
Release X.Y
4 months
ITK
Release X.Y.3
6 months
CMake
CTest
CPack
Dashboard
12
National Alliance for Medical Image Computing
http://na-mic.org
NA-MIC Software Process
• Emphasizes Cross Platform Testing
• Provides informal
– Requirements specification
– Design documents
• Which are captured via collaboration tools
– Tcon, email, wiki, software documentation
• Traceability
– Implicit in logs and collaboration
13
National Alliance for Medical Image Computing
http://na-mic.org
What is 3D Slicer?
• A platform for image analysis and
visualization
• Current Releases 2.7 and 3.2
– 2.x most features and
documentation
– 3.x focus of current activity
• A freely-downloadable program
– Source code and executables
available for Windows, Linux, and
Mac OS X
• Slicer is a research platform:
– NOT an FDA approved medical
device
– NOT finished – some parts will
work better than others
14
National Alliance for Medical Image Computing
http://na-mic.org
Slicer 3.2
May 2008
Kitware, Inc.
GE Research
Isomics, Inc.
BWH
UCLA
UCSD
U Iowa
MIT
GaTech
UNC
U Utah
MGH
UCI
Harvard
WashU…
NAC
NCIGT
NA-MIC
mBIRN
fBIRN…
http://slicer.org
National Alliance for Medical Image Computing
http://na-mic.org
3D Slicer Numbers
•
Numbers May 2007:
Subversion Commits: 3,407
Lines of Code*: 371,428
Bugs & Features:
154 Submitted
63 Closed
Active Developers with svn
access†: 33
Numbers July 2008:
Subversion Commits: 7303
Lines of Code*: 448,175
Developers: 42
*: find . -iname \*.h -o -iname \*.cxx -o -iname \*.tcl -o -iname \*.java -o -name \*.py | grep -v svn | xargs wc
†: svn log | grep "^r" | cut -d " " -f 3 | sort | uniq | wc
16
National Alliance for Medical Image Computing
http://na-mic.org
Image provided by A. Golby, F. Talos, P. Black
Slicer3 “Observer MVC”
Pattern
•
•
•
MRML (Model)
– For Scene Description and Application State
– MRML Nodes are Persistent and Undoable
– Scene and Nodes are Observable
– XML Serialization, Undo/Redo
Logic Encapsulate VTK and ITK Pipelines (Controller)
– Observe MRML to Configure Pipelines
– Help Create/Manage Nodes
– No UI Components (no Widgets, Actors, Mappers,
Renderers or RenderWindows)
GUI (View)
– Observe and Edit MRML
– Interact with User and Display Hardware
GUI
Renderers
Widgets
Edit
Observe
Logic
Observe
Edit
Observe
Edit
MRML Nodes
“Observe” means generic event mechanisms
are used to pass information.
“Edit” means code can directly call methods.
Example: GUI can call methods in Logic classes,
but Logic cannot call GUI methods.
MRML cannot call Logic or GUI methods.
There can be many observers for any event.
17
National Alliance for Medical Image Computing
http://na-mic.org
Scene Description
18
National Alliance for Medical Image Computing
http://na-mic.org
Provided by D. Gering
Image/Scene Management
• XML-Based MRML File Stores
Scene Description
– Volumes (Images,
Label Maps)
– Models
– Hierarchical Affine
Transforms
– Scene Data (Cameras,
Colors, Fiducials, etc).
• Manipulated in World
Coordinates based on Patient
RAS
19
National Alliance for Medical Image Computing
http://na-mic.org
Provided by S. Pieper
Provided by Archip, Warfield
Neurosurgery Example
Alignment of all pre-operative
datasets to the intra-operative
images achieved during the
neurosurgery.
Archip N, Clatz O, Whalen S, Kacher
D, Fedorov A, Kot A, Chrisochoides N,
Jolesz F, Golby A, Black PM, Warfield
SK. Non-rigid alignment of preoperative MRI, fMRI, and DT-MRI with
intra-operative MRI for enhanced
visualization and navigation in imageguided neurosurgery. Neuroimage.
2007 Apr 1;35(2):609-24
20
National Alliance for Medical Image Computing
http://na-mic.org
Modules
21
National Alliance for Medical Image Computing
http://na-mic.org
Slicer3 Command Line
Modules
Common architecture for
interactive and batch
processing
Slices Courtesy Jim Miller
National Alliance for Medical Image Computing
http://na-mic.org
Each module has …
… an entry in the module menu
… a panel of user interface controls
National Alliance for Medical Image Computing
http://na-mic.org
Slicer Modules
• Interactive Modules
– Interact Closely with GUI
– Volume Rendering, Models, Editor…
• Command Line Modules
– “Batch Mode” (Can be run from command line or script)
– Registration, Filtering, DTI Tractography…
• Both Interactive and Command Line Modules are Dynamically
Loaded
– They can be Built and Distributed Independent of Slicer
• Today we will look in detail only at Command Line Modules
National Alliance for Medical Image Computing
http://na-mic.org
EM Interactive Module
MRML
•Global Parameters
•Hierarchical
parameters
•Image data
•Segmentation output
25
Logic
•Manage MRML
nodes
•API for access to
parameters
•Manage
hierarchy
•Segmentation
algorithm
National Alliance for Medical Image Computing
http://na-mic.org
Provided by B. Davis
GUI
•Window to parameter
set
•Wizard
•Interaction with
images
Command Line Module
MRML
•Global Parameters
•Hierarchical
parameters
•Image data
•Segmentation output
26
Logic
•Manage MRML
nodes
•API for access to
parameters
•Manage
hierarchy
•Segmentation
algorithm
National Alliance for Medical Image Computing
http://na-mic.org
Provided by B. Davis
CLI
•XML Parameter File
•File I/O
•Batchable
Existing Command Line
Modules
•
Demonstration
–
–
•
Converters
–
–
–
–
–
•
Execution Model Tour
Your new hello world module
Create a DICOM Series
Dicom DWI loader
Dicom to Nrrd
GE Dicom to NRRD Converter
Orient Images
Filtering
–
–
–
–
–
–
–
–
–
–
–
Calculate Volume Statistics
CheckerBoard Filter
Extract Skeleton
Histogram Matching
Otsu Threshold
Resample Volume/ Resample Volume 2
Voting Binary Hole Filling
Zero Crossing Based Edge Detection Filter
Filtering.Arithmetic:
Add Images
Subtract Images
National Alliance for Medical Image Computing
http://na-mic.org
Existing Command Line
Modules
•
Filtering.Denoising
–
–
–
–
•
Segmentation
–
–
–
–
•
Grayscale Fill Hole
Grayscale Grind Peak
Model Generation
–
–
–
–
–
•
EMSegment Simple
FreesurferSurfaceSectionExtraction
Otsu Threshold Segmentation
Simple region growing
Filtering.Morphology
–
–
•
Curvature Anisotropic Diffusion
Gaussian Blur
Gradient Anisotropic Diffusion
Median Filter
Grayscale Model Maker
Label Map Smoothing
Model Maker
Multiple models example
Probe Volume With Model (Paint)
Meshing.VoxelMesh
–
Voxel Meshing
National Alliance for Medical Image Computing
http://na-mic.org
Registration Command
Line Modules
•
Registration
–
–
–
–
–
Affine registration
Deformable BSpline registration
(Utah) Deformable BSpline registration
Linear registration
RealignVolume
National Alliance for Medical Image Computing
http://na-mic.org
Diffusion Command Line
Modules
•
Diffusion Tensor
–
•
•
•
•
•
•
•
Resample DTI Volume
Diffusion Tensor Estimation
Diffusion Tensor Scalar Measurements
Simple DWI IO Test
Rician LMMSE Image Filter
Tractography.Editor: ROISelect
Tractography.Seeding
Stochastic Tractography
–
–
–
Generate Connectivity Map
Stochastic ROI Tract Filter
Stochastic Stochastic Tractography Filter
National Alliance for Medical Image Computing
http://na-mic.org
Example Modules
http://www.nitrc.org/projects/slicer3examples/
http://www.na-mic.org/Wiki/index.php/Slicer3.2:Training
National Alliance for Medical Image Computing
http://na-mic.org
Parameters
<integer> | <float> | <double> | <boolean> | <string> |
<integer-vector> | <float-vector> | <double-vector> | <string-vector> |
<integer-enumeration> | <float-enumeration> | <double-enumeration> | <string-enumeration> |
<file> | <directory> |
<image>[type="scalar|label|tensor|diffusion-weighted|vector|model"] |
<geometry> [type="fiberbundle|model"] |
<point>[multiple="true|false"] [coordinateSystem="lps|ras|ijk"] |
<region>[multiple="true|false"] [coordinateSystem="lps|ras|ijk"]
•
•
GUI Automatically Generated from
XML
C++ Argument Parsing Code
Automatically Generated from XML
National Alliance for Medical Image Computing
http://na-mic.org
Parameter Example
<?xml version="1.0" encoding="utf-8"?>
<executable> <description> HelloWorld Example</description>
…
<parameters>
<label>Input/Output</label>
<description>Input/output parameters</description>
<image>
<name>helloSlicerInputVolume</name>
<label>Input Volume</label>
<channel>input</channel>
<index>0</index>
<default>None</default>
<description>Input volume</description>
</image>
…
</parameters>
<parameters>
<label>Discrete Gaussian Parameters</label>
<description>Parameters of the Discrete Gaussian Filter </description>
<double>
<name>variance</name>
<longflag>--variance</longflag>
<description>Variance ( width of the filter kernel) </description>
<label>Variance</label>
<default>0.5</default>
</double>
</parameters>
</executable>
33
National Alliance for Medical Image Computing
http://na-mic.org
C++ Module
#include <iostream>
#include "HelloSlicerCLP.h"
#include "itkImage.h"
#include "itkImageFileReader.h"
#include "itkImageFileWriter.h"
#include "itkDiscreteGaussianImageFilter.h"
int main(int argc, char * argv []) {
PARSE_ARGS;
typedef itk::Image< short, 3 > ImageType;
typedef itk::ImageFileReader< ImageType > ReaderType;
typedef itk::ImageFileWriter< ImageType > WriterType;
ReaderType::Pointer reader = ReaderType::New();
WriterType::Pointer writer = WriterType::New();
reader->SetFileName( helloSlicerInputVolume.c_str() );
writer->SetFileName (helloSlicerOutputVolume.c_str());
typedef itk::DiscreteGaussianImageFilter <ImageType, ImageType> FilterType;
FilterType::Pointer filter = FilterType::New();
try {
filter->SetInput(reader->GetOutput());
filter->SetVariance(variance);
writer->SetInput(filter->GetOutput());
writer->Update();
}
catch (itk::ExceptionObject &excep) {
std::cerr << argv[0] << ": exception caught !" << std::endl; return EXIT_FAILURE;
}
return EXIT_SUCCESS;
}
34
National Alliance for Medical Image Computing
http://na-mic.org
Python Module
XML = """<?xml version="1.0" encoding="utf-8"?>
<executable>
<category>Filtering.Denoising</category>
...
def toXML(): return XML;
def Execute ( inputVolume, outputVolume, conductance=1.0, timeStep=0.0625, iterations=1 ):
print "Executing Python Demo Application!"
Slicer = __import__ ( "Slicer" );
slicer = Slicer.Slicer()
in = slicer.MRMLScene.GetNodeByID ( inputVolume );
out = slicer.MRMLScene.GetNodeByID ( outputVolume );
filter = slicer.vtkITKGradientAnisotropicDiffusionImageFilter.New()
filter.SetConductanceParameter ( conductance )
filter.SetTimeStep ( timeStep )
filter.SetNumberOfIterations ( iterations )
filter.SetInput ( in.GetImageData() )
filter.Update()
out.SetAndObserveImageData(filter.GetOutput())
return
National Alliance for Medical Image Computing
http://na-mic.org
Behind the scenes
•
•
Tasks queued for processing thread
Three types of modules:
–
–
–
•
•
•
executable,
shared object, and
python modules
Scalar images sent via files for
executables and memory* for
shared objects and python
Vector images, tensor images,
geometry, tables, transforms sent
via files
Scalars, file names, directories,
fiducials, regions sent via command
line
* VTK-based modules using scalar images are only
supported as executable (command line) modules.
National Alliance for Medical Image Computing
http://na-mic.org
Population Statistics
37
National Alliance for Medical Image Computing
http://na-mic.org
BIRN Integration
distortion
correction
tools
acquisition
protocols
local
databases
PubMed
IBVD
.
.
.
Visualization
&
Interpretation
BrainInfo
(.xcat)
(.xar)
BIRN
workflows
analysis
tools
population
statistics
National Alliance for Medical Image Computing
http://na-mic.org
mBIRN Informatics
• BIRN Data Repository (BDR)
• eXtensible Neuroimaging
Archive Tool (XNAT)
http://www.xnat.org
• XML-Based Clinical
Experiment Data Exchange
Schema (XCEDE)
http://www.xcede.org
• Stores Images,
Demographics, Clinical Data,
Analysis Results
• 3D Slicer Interoperability
(currently read-only, read/write
prototype exists)
National Alliance for Medical Image Computing
http://na-mic.org
Query Atlas Module
• BIRN Scientific Interpretation Tool
– Compatible with fMRI and Morphometry Datasets in BIRNStandard Formats
• Provides a Link Between Images and Text Databases
– Uses Anatomic Labeling + Ontologies
– Links to Definitions, Publications, Quantifications
– E.g. Wikipedia, PubMed, IBVD, BrainInfo
National Alliance for Medical Image Computing
http://na-mic.org
Query Atlas Features
• Input:.xcat or .qdec from .xar
• Hardware Accelerated
Interactive 3D Annotation
• Ontology Engine
– FreeSurfer, UMLS, BIRNLex,
NeuroNames, IBVD
– Interactive Browser
• Direct Browser Launch to
Search Sites using Selected
Terms
National Alliance for Medical Image Computing
http://na-mic.org
BIRN Population Statistics
• Query, Design,
Estimate, Contrast of
Population Statistics
– XNAT to Select
Subjects
– FreeSurfer QDEC
Runs on Server
– .QDEC file in .XAR
Web Download
– Slicer3 Interactive
Visualization
– Integrated with Query
Atlas
National Alliance for Medical Image Computing
http://na-mic.org
Community
43
National Alliance for Medical Image Computing
http://na-mic.org
Overall Goals
• Why Medical Research?
– To Help Patients!
44
National Alliance for Medical Image Computing
http://na-mic.org
NA-MIC Roots: NIH
• Roadmap Initiative
• http://grants1.nih.gov/grants/guide/rfa-files/RFARM-04-003.html
– …will create a networked national effort to
build the computational infrastructure for
biomedical computing in the nation…
– The establishment of the NIH NCBC was
called for in the Biomedical Information
Science and Technology Initiative report in
1999
45
National Alliance for Medical Image Computing
http://na-mic.org
NA-MIC Governance
Software Sharing:
http://grants1.nih.gov/grants/guide/rfa-files/RFA-RM-04-003.html
• …software should be freely available …
• …permit the commercialization of
enhanced or customized versions …
• …include the ability of researchers outside
the center and its collaborating projects to
modify the source code and to share
modifications …
46
National Alliance for Medical Image Computing
http://na-mic.org
NA-MIC Kit Governance
•
•
•
•
•
•
•
Free, BSD style Open-Source
NA-MIC Provides “Reference Standard Distribution”
– Maintain Official Releases
– Avoid Code Forking, Avoid Non-Free Code Creeping into Core
No “Knowing” Addition of Patented Techniques into NA-MIC Kit
Technology Base for Suited for both Research and Commercialization
– Okay to build GPL or Closed SW on top of NA-MIC Kit
– But no GPL or Closed SW in NA-MIC Kit
Emerging community process for contributions http://insight-journal.org
Support Required for Contributions (no “shoot and forget”)
Copyright to Substantive Works Remains with Creators
– Available under NA-MIC Approved Licenses for Distribution with Kit
47
National Alliance for Medical Image Computing
http://na-mic.org
FOSS – A Public Highway…
• Open-source is like a Public
Road System
– Provides Infrastructure
for a Variety of Uses
– Driveways can Lead to
Anything:
• a Public Park
• a Private Facility
FOSS= Free Open Source Software
48
National Alliance for Medical Image Computing
http://na-mic.org
Provided by Pieper, Kikinis
NA-MIC Policy
NA-MIC Kit
49
National Alliance for Medical Image Computing
http://na-mic.org
Industry
NA-MIC: A Network of Peers
•
•
•
•
•
Leadership:
– BWH: Ron Kikinis, (Overall PI)
Core 1 Algorithms
– Utah: Ross Whitaker (Core 1 PI)
– MIT: Eric Grimson
– UNC: Guido Gerig
– MGH: Bruce Fischl, Dave Kennedy
– GaTech: Allen Tannenbaum
Core 2 Engineering
– Kitware: Will Schroeder (Core 2 PI)
– GE: Jim Miller
– Isomics: Steve Pieper
– UCSD: Mark Ellisman
– UCLA: Art Toga
– WashU: Dan Marcus
Core 3 DBP 2004-2007
– BWH: Martha Shenton
– Dartmouth: Andy Saykin
– UCI: Steve Potkin
– UofT: Jim Kennedy
DBP 2007
– UNC: H. Cody
– BWH: M. Kubicki
– Mind Institute: J. Bockolt
– Queens University: G. Fichtinger
50
•
•
•
•
National Alliance for Medical Image Computing
http://na-mic.org
Core 4 Service
– Kitware: Will Schroeder
Core 5 Training
– MGH: Randy Gollub
Core 6 Dissemination
– Isomics: Steve Pieper, Tina Kapur
Core 7 Management
– BWH: S. Manandhar
Provided by Pieper, Kikinis
Communication





Daily – e-mail, dashboards, wikis
Weekly – telephone conferences
Periodic – architecture reviews, workshops
Semi-annual– Programmer/Project week
Yearly – All Hands Meeting
51
National Alliance for Medical Image Computing
http://na-mic.org
All Hands Meeting
52
National Alliance for Medical Image Computing
http://na-mic.org
Programmer’s Week
53
National Alliance for Medical Image Computing
http://na-mic.org
NA-MIC Project Week
• June 23-27, 2008
– 120 Participants
– Dozens of Projects
– Hands On Software
Development
National Alliance for Medical Image Computing
http://na-mic.org
Dissemination and Training
•
•
•
•
National and International
Events
– MIT, MGH, UNC,
EPFL, NIH, UNM,
UCSD…
All Materials on Wiki
Project Weeks
– Full Week Each
Summer
– ½ Week at Winter
AHM
Workshops
– MICCAI 2005, 2006,
2007, 2008..
– OHBM, RSNA,
Munich, NCI…
55
National Alliance for Medical Image Computing
http://na-mic.org
Training Courses
• June In Munich
– 2 Packed Days, 40 Participants
– Cover Spectrum of Use and
Development
• Data Visualization
• Registration
• Tracked Surgical Instruments
• Developing Custom Modules
• Upcoming
– NIH/NCI, Robarts Inst., Stanford,
MICCAI, RSNA, WashU
• > 1,000 Participants Since 2004
National Alliance for Medical Image Computing
http://na-mic.org
NAMIC Training Portfolio
http://www.na-mic.org/Wiki/index.php/Slicer:Workshops:User_Training_101
• Google for “Slicer
101”
• Most training
material is for
Slicer2.x with
increasing amounts
of Slicer3 content
National Alliance for Medical Image Computing
http://na-mic.org
It’s all great, but…
• A few caveats to keep in mind
• It’s a big, distributed group
– Good spin: “Wisdom of the Crowds”
– Bad spin: “Designed by Committee”
• It’s all research
– Constantly evolving
– Never enough documentation
– Plurality of approaches
• The net result is overwhelmingly positive.
58
National Alliance for Medical Image Computing
http://na-mic.org
Getting Involved
•
•
•
Winter Project Week 2009
– January, Salt Lake City, UT
Summer Project Week 2009
– June at MIT Stata Center, Cambridge MA
Collaborations PAR-05-063
– Automated FE Mesh Development
N. Grosland PI, U Iowa + Isomics, Inc.
– Measuring Alcohol and Stress Interaction with Structural and Perfusion MRI
• J. Daunais PI, Wake Forest University and Virginia Polytechnic Institute + K.
Pohl BWH
– An Integrated System for Image-Guided Radiofrequency Ablation of Liver
Tumors
• K. Cleary PI, Georgetown + N. Hata BWH
– http://www.na-mic.org/Wiki/index.php/Collaborator:Resources
59
National Alliance for Medical Image Computing
http://na-mic.org
Summary
• End-to-End Platform for Translational Image Research
• Open Source and Extensible
• Standard Methodologies Facilitate Sharing and Communication
60
National Alliance for Medical Image Computing
http://na-mic.org
More Information
• SPL website:
http://www.spl.harvard.edu
• NA-MIC wiki:
http://wiki.na-mic.org
• 3D Slicer
http://www.slicer.org
61
National Alliance for Medical Image Computing
http://na-mic.org