Image Registration
Lecture 1: Introduction
February 22, 2004
Prof. Charlene Tsai
http://www.cs.ccu.edu.tw/~tsaic/teaching/spring2005_grad/main.html
Outline
 Syllabus
 Registration problem
 Applications of registration
 Components of a solution
 Thematic questions underlying registration
 Software toolkits
Image Registration
Lecture 1
2
Syllabus - Topic
 Image registration:
 Determining the mapping between two images of
the same object, similar objects, the same region
or similar regions
 All aspects of the problem will be covered:




Underlying mathematics
Images
Algorithms
Implementations
 Applications
 Special emphasizis on software toolkits
Image Registration
Lecture 1
3
Syllabus - Hours
 Lecture hours:
 Tuesday and Thursday
 Office hours
 By appointment
Image Registration
Lecture 1
4
Syllabus - Prerequisites
 Data structures
 Calculus
 Linear algebra:
 Vectors and matrices
 Experience working with images
 C++ programming experience
 Templates!
Image Registration
Lecture 1
5
Syllabus - Requirements
 50% - Weekly homework assignments and
programming projects (possibly 8)
 25% - Extended programming project (due
before 23:59 of May 15)
 25% - 10-page research paper (due before
23:59 of June 15)
 No examines!
 Late assignments will not be accepted without
prior arrangement or a verified personal
emergency
Image Registration
Lecture 1
6
Syllabus - Course Materials
 Powerpoint lectures will be placed on the
course website
 Software toolkits will include tutorials
 Reading materials, mostly journal papers, will
also be placed on the website
 Most lecture slides by courtesy of Prof Chuck
Stewart from RPI and Dr. Luis Ibanez from
Kitware.
Image Registration
Lecture 1
7
Syllabus - Topics











Introduction
Mathematical background
First examples
Intensity-based registration and ITK
Feature-based registration and the RPI toolkit
Initialization techniques
Multiresolution techniques
Mutual information
Video registration and image mosaics
Deformable registration
Project presentation
Image Registration
Lecture 1
8
Syllabus - Academic Integrity
 Students may discuss homework and
programming assignments
 Solutions must be written in students’ own
words
 Extended programming project and research
paper must be individual work with
appropriate citations
 A serious incident will result in failing the
course
Image Registration
Lecture 1
9
Registration Problem Definition
q = (912,632)
p = (825,856)
q = T(p;a)
Pixel location in first image
Homologous pixel location in
second image
Pixel location mapping function
Image Registration
Lecture 1
10
Example Mapping Function
q = (912,632)
p = (825,856)
Pixel scaling and
translation
Image Registration
Lecture 1
11
Registration Problem Definition
p = (825,856)
q = T(p;q)
Problems:
• Form of mapping function T
• Unknown mapping parameters q
• Unknown correspondences, p,q
Image Registration
Lecture 1
q = (912,632)
“Chicken-and-egg”
problem
12
Applications: Multimodal Integration
 Two or more different sensors view same region or
volume
 Different viewpoints
 (Some specialized sensors have two or more
coincident modalities, so registration is not needed.)
 Different information is prominent in each image
 The images may even have different dimensions!
 Range images vs. intensity images
 CT volumes vs. fluoro images
Image Registration
Lecture 1
13
Example: MR-CT Brain Registration
MR
CT
 MR (magnetic
resonance) measures
water content
 CT measures x-ray
absorption
 Bone is brightest in CT
and darkest in MR
 Both images are 3d
volumes
Source: http://www-ipg.umds.ac.uk/d.hill/hhh/10/10.pdf
Image Registration
Lecture 1
14
MR-CT Registration Results
Aligned images
Image Registration
Superimposed images, with bone
structures from CT in green
Lecture 1
15
Retinal Angiogram and Color Image
Image Registration
Lecture 1
16
Applications: Image Mosaics
 Many, partially overlapping images
 No one gives a complete view
 Goal: “stitch” images together
 Requires:
 Limited camera viewpoint such as rotation about
optical center
 Simple surface geometry such as plane or
quadratic
Image Registration
Lecture 1
17
Retinal Image Mosaics
Image Registration
Lecture 1
18
Sea-Floor Mosaics
Courtesy Woods Hole Oceanographic Institution
Image Registration
Lecture 1
19
Spherical Mosaics
Images from Sarnoff Corporation
Image Registration
Lecture 1
20
Applications: Building 3d Models
 Range scanners store an (x,y,z)
measurement at each pixel location
 Each “range image” gives a partial view
 Must register range images and texture map
them
 Applications:
 Reverse engineering
 Digital architecture and archaeology
Image Registration
Lecture 1
21
Examples
http://www1.cs.columbia.edu/~allen/NEW/workshop.html
Image Registration
Lecture 1
22
Applications: Change Detection
 Images taken at different times
 Following registration, the differences
between the images may be indicative of
change
 Deciding if the change is really there may be
quite difficult
Image Registration
Lecture 1
23
Retinal Change Example
Image Registration
Lecture 1
24
Regions Showing Change
Image Registration
Lecture 1
25
Applications: Video Super-Imposed on 3d Model
Taken from Sarnoff Corporation research
Image Registration
Lecture 1
26
Other Applications
 Multi-subject registration to develop organ
variation atlases.
 Used as the basis for detecting abnormal
variations
 Object recognition - alignment of object model
instance and image of unknown object
 Industrial inspection
 Compare CAD model to instance of part to
determine errors in manufacturing process
Image Registration
Lecture 1
27
Steps Toward a Solution
 Analyze the images
 Determine the appropriate image primitives
 Determine the transformation model
 Geometric and intensity
 Design an initialization technique
 Develop constraints and an error metric on
the transformation estimate
 Design a minimization algorithm
 Develop a convergence criteria
Image Registration
Lecture 1
28
Software Toolkits
 ITK
 Medical image processing, segmentation, and
registration toolkit
 C++, heavily templated, data flow architecture
 Registration stresses intensity-based approaches
 VXL
 Computer vision applications
 C++, moderate templating
 Registration stresses feature-based approaches
Image Registration
Lecture 1
29
Summary: Pervasive Questions
 Three questions to consider in approaching
any registration problem:
 What intensity information or image
structures is/are consistent between the
images to be registered?
 What is the geometric relationship between
the image coordinate systems?
 What prior information can be used to
constrain the domain of possible
transformations?
Image Registration
Lecture 1
30
Looking Ahead: Lecture 2 - Friday, January 16
 Mathematical background, part 1:
 Vectors and matrices
Image Registration
Lecture 1
31
Homework Problem
 Due Tuesday, March 1st before class (via
email to me)
 Problem:
 Find an application of registration,
preferably in a research area of interest to
you. In a short write-up (less than a full
page), describe the problem and attempt to
sketch answers to the three “Pervasive
Questions” posed.
Image Registration
Lecture 1
32