Image Management
Dr. Hayit Greenspan
Dept of BioMedical Engineering
Faculty of Engineering
hayit@eng.tau.ac.il
640-7398
Roles for Imaging
in Health Care:
Diagnosis
Assessment and Planning
Guidance of Procedures
Communication
Education and Training
Research
Image
Diagnosis in
Dermatology
Fetus Ultrasound
Example of cross-sections through several parts of the body: skull, thorax, and
abdomen,
obtained by computed tomography.
Visualization of the values of the attenuation coefficients by way
of gray values produces an anatomic image.
Spinal cord
Brain section
MRI Image Diagnosis
Roles for Imaging
in Health Care:
Diagnosis
Assessment and Planning
Guidance of Procedures
Communication
Education and Training
Research
fMRI
A functional map (in color) in the cerebellum during performance of a cognitive pegboard puzzle task, overlaid on a T2*-weighted axial image in gray scale. The dentate
nuclei appear as dark crescent shapes at the middle of the cerebellum due to iron
deposits. fMRI images were acquired by conventional T2*-weighted FLASH techniques
with a spatial resolution of 1.25x1.25x8 mm3 and a temporal resolution of 8 seconds.
Each color represents a 1% increment, starting at 1%. R, right cerebellum; L, left
cerebellum. A left-handed subject used the left hand to perform the task. Bilateral
activation in the dentate nuclei and cerebellar cortex was observed. The activated area
in the dentate nuclei during performance of pegboard puzzle was 3-4 times greater than
that seen during the visually guided peg movements. (see details in Kim et al., 1994b).
fMRI
Whole brain functional imaging study during a visuo-motor error detection and correction task.
Functional images were acquired by the multi-slice single-shot EPI imaging technique with
spatial resolution of 3.1x3.1x5 and temporal resolution of 3.5 seconds. The skull and associated
muscles were eliminated by image segmentation. The 3-D image constructed from multi-slice
images was rendered by Voxel View program (Vital Images, Fairfield, Iowa).The task was to
move a cursor from the central start box onto a square target by moving a joystick. Eight targets
were arranged circumferentially at 450 angles and displaced radially at 200 around a central start
box. Activation (in color) is observed at various brain areas. Top image displays the brain as a
3-D solid object so that only the cortical surface is seen. In the bottom image, a posterior section
was removed at the level of the associative visual cortex to display activation not visible from the
surface (Kindly provided by Jutta Ellermann, Jeol Seagal, and Timothy Ebner).
Medical Image Databases
•
Medical Images are at the heart of diagnosis, therapy and follow-up.
•
Digital medical image data in US per year:
1015 bytes (petabytes).
•
Generation & Acquisition 
Post processing & Management.
•
Medical imaging information types:
still images; pictures; moving images; structured text; plain text; sound;
graphics.
•
Driving the shift toward multimedia applications in medical imaging:
market demand; capital investment in imaging devices; need to
organize and store multimodal image data + associated clinical data;
ability to extract info in images.
Biomedical Imaging
Structural
MRI
Functional
Ultrasound
fMRI
X-ray CT
Microscopy
Projectional
x-ray
Emission
CT
Medical
optical
imaging
CR
Mammograph
PET
DSA
Modality
Image
Dimension
(pixels)
Gray Level
(bits)
Avg. Size
(Mbytes)
MRI
256x256
12
8-20
Ultrasound
512x512
8
5-10
DSA
(per run)
512x512or
1024x1024
8
100-500
SPECT
Current Information Systems
Originators
Publishers
Libraries
Users
Digital Libraries
Originators
Repositories
Service X
Value-added
Index Services
Users
Multimedia Information Systems:
Work-centered Scenario
Maps Legacy Documents
Photos
Databases
Other
Collections
Co-workers/
Collaborators
Visual Information Systems
Example:
Patient needs neurosurgery to remove a tumor
–
–
–
–
–
CT, MRI, PET scans: digitized and scanned
Images are registered with a 3D brain model
Locate tumor
Path planning
Using tumor as template, request to find:
• patients of same sex
• with similar tumors
• in similar positions
Imaging Informatics
•
Information systems and networks that facilitate the
Acquisition
Storage
Transmission
Processing
Analysis
Management
of medical images.
• Imaging Informatics- a new discipline:
Image generation
Image management
Image manipulation
Image integration
Basic concepts in
Image Manipulation
•
•
•
•
Global Processing: enhance contrast resolution;
Segmentation:
finding regions of interest;
Feature detection & extraction;
Classification;
Examples:
•
•
•
•
•
•
Histogram equalization
Temporal subtraction (DSA)
Screening
Quantitation
3D reconstruction and visualization
Multimodality image fusion
Contrast enhancement
Principle of contrast enhancement:
(a) intensity distribution along a line of an image;
(b) same distribution after injection of the contrast medium;
(c) intensity distribution
after subtraction;
(d) intensity distribution after contrast enhancement.
Example of digital subtraction angiography (DSA) of the
bifurcation of the aorta
An initial image mask is obtained digitized and stored
Contrast medium is injected
Number of images are obtained.
Mask is subtracted
The resulting image contains only the relevant information
The differences can be amplified so the eye will be able to perceive the the blood
vessels.
Quality of deteriorate due to movements of the body can be corrected to some ex
Texture Segmentation of MRI images
VOXEL-MAN(Hamburg): 3D Visualization
http://www.uke.uni-hamburg.de/institute/imdm/idv/index.en.html
Atlasas of brain and
other organs: allow
views from any
viewpoint;
Fusion of modalities
+Anatomical atlases
Video: COVIRA
Computer Vision in Radiology
Basic concepts in
Image Management
•
Digital acquisition of images offers the exciting prospect of reducing the
physical space requirements, material cost, and manual labor of
traditional film-handling tasks, through online digital archiving, rapid
retrieval of images via querying of image databases, and high-speed
transmission over communication networks.
•
Researchers are working to develop such systems that have such
capabilities - picture archiving and communication systems (PACS).
•
Issues that need to be addressed for PACS to be practical:
–
–
–
–
–
–
technology for high-resolution acquisition
high capacity storage
high-speed networking
standardization of image-transmission and storage formats
storage management schemes for enormous volumes of data
design of display consoles/workstations
Evolution of Image Management in PACS
•
Early attempts in mid 80s
– Univ. of Kansas, Templeton et al (84): earliest prototype systems to study
PACS in radiology
– Inst of radiology in St. Louis, Blaine et al (83): PACS Workbench
experiments in image acquisition, transmission, archiving and viewing
•
Substantial progress on several fronts:
– Standards (DICOM) support transition from acquisition devices to storage
devices
– Expansion in disk capacities and dramatic decreases in cost
– Hierarchical storage-management schemes
– Compression methods
– Increased resolution workstation display
– Image manipulation tools
•
Many Departments have mini-PACS; Large scale PACS increased in
number from 13 to 23 in a 15-month period.
Image Management:
Indexing & Retrieval
We formed image archives
How do we access the content??
Extract content from file headers
Add Keywords
***Content-based Image Retrieval***
Visual Information Systems
Visual Information
Feature types
Color, texture
shape...
Which features should we use?
How are we to organize them?
Prioritize?
Arrange for Search?
Global Histograms
Local Regions
Trees...
Examples of
search queries
Search for:
“Example like this”
“similar image features”
“50% blue and 50% green”
Visual Representation
• Text/Keywords wont do it:
“ One picture is worth a thousand words”
• Standard Object Recognition wont do it
• Our Representation & Indexing Goals
– retrieve visual data based on content
– domain independent
– automated
Image Representation
• Image Processing
• Computer Vision
• Image Representation: Pixels to Content
Image Similarity
Multimedia
Object
Insertion
Query
Multimedia
Object
Feature Processing Module
Calculate Similarity
Stored Features
Query Features
Storage and Retrieval of Images and
Video
User Interface
Content-Based
Retrieval
Organization
Database
Management
Metadata
Database
Content-based Information Retrieval
Image
Pre-Processing
Scene Change
Detection
Camera &
Object Motion
Key-Frame
Extraction
Feature Extraction & Representation
Camera
Motion
Object
Motion
Object
Color
Sketch
Shape
Texture
Spatial
Relationships
Organization Module:
• Efficient query processing necessitates organization of indices
for efficient search
• Image/Video indices:
– are approximate
– interrelated multiple attributes
– not ordered
• Need flexible data structures (quad-tree, R-tree..)
Database Management Module
Physical storage structure and access path to the database
• insulation between programs and data
• provides a representation of the data
• supprots multiple views of data
• ensures data consistency
Video: Image Guided
Decision Support System for
Pathology, Univ. of Rutgers
Evaluation Criteria for
Image Retrieval Systems:
Automation
Multimedia Features
Adaptability
Abstraction
Generality
Content Collection
Categorization
Compressed Domain
Networked Multimedia for Medical Imaging
Radiology Informatics Lab,
Univ. of San Francisco
Multimedia
application 1
Multimedia
application 2
Multimedia
application N
Medical Image DBMS
Data sources
Postprocessing
Visualization
Communication
Networked Multimedia for Medical Imaging
Radiology Informatics Lab,
Univ. of San Francisco
Multimedia Medical Imaging Applications testbed:
•
•
•
•
Bone age assessment
Temporal lung node analysis
Collaborative image consultation
Noninvasive neurosurgical planning