Mixed-Integer Programming Formulation of the
Registration Problem using Assignment Problems
with an Application in Phoniatrics
Michael Stiglmayr Raphael Schwarz Kathrin Klamroth
Günter Leugering Jörg Lohscheller
Institute of Applied Mathematics
& Department of Phoniatrics and Pediatric Audiology
Friedrich-Alexander-University Erlangen-Nürnberg
(partially supported by Deutsche Forschungsgemeinschaft, SFB 603)
Dagstuhl Seminar on
Sensor Data and Information Fusion
in Computer Vision and Medicine
Juli 30 – August 4, 2006
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Point Based Registration Algorithms
principle structure
◮
find corresponding points (assignment)
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Point Based Registration Algorithms
principle structure
◮
find corresponding points (assignment)
◮
find a transformation function, such that the “distance” of
assigned points is minimized
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Point Based Registration Algorithms
principle structure
◮
find corresponding points (assignment)
◮
find a transformation function, such that the “distance” of
assigned points is minimized
existing methods
◮
landmark based algorithms (known assignment)
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Point Based Registration Algorithms
principle structure
◮
find corresponding points (assignment)
◮
find a transformation function, such that the “distance” of
assigned points is minimized
existing methods
◮
landmark based algorithms (known assignment)
◮
iterative closest point
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Point Based Registration Algorithms
principle structure
◮
find corresponding points (assignment)
◮
find a transformation function, such that the “distance” of
assigned points is minimized
existing methods
◮
landmark based algorithms (known assignment)
◮
iterative closest point
◮
robust point matching
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Robust Point Matching (RPM)
[Chui and Rangarajan, 2003]
X = {x1 , . . . , xI } and Y = {y1 , . . . , yJ } are sets of corresponding
points in the template and the reference image.
min
z,f
s.t.
Michael Stiglmayr
I P
J
P
2
2
zij kyj − f (xi )k + λ kLf k − ξ
i=1 j=1
PI +1
i=1 zij
PJ+1
j=1 zij
I P
J
P
zij
i=1 j=1
=
=
zij ∈
f ∈
1
1
{0, 1}
F
∀j ∈ {1, . . . , J}
∀i ∈ {1, . . . , I }
∀i ∈ {1, . . . , I +1} ∧ j ∈ {1, . . . , J +1}
Mixed-integer programming for registration
University Erlangen-Nürnberg
Robust Point Matching (RPM)
[Chui and Rangarajan, 2003]
X = {x1 , . . . , xI } and Y = {y1 , . . . , yJ } are sets of corresponding
points in the template and the reference image.
min
z,f
s.t.
Michael Stiglmayr
I P
J
P
2
2
zij kyj − f (xi )k + λ kLf k − ξ
i=1 j=1
PI +1
i=1 zij
PJ+1
j=1 zij
I P
J
P
zij
i=1 j=1
=
=
zij ∈
f ∈
1
1
{0, 1}
F
∀j ∈ {1, . . . , J}
∀i ∈ {1, . . . , I }
∀i ∈ {1, . . . , I +1} ∧ j ∈ {1, . . . , J +1}
Mixed-integer programming for registration
University Erlangen-Nürnberg
Robust Point Matching (RPM)
[Chui and Rangarajan, 2003]
X = {x1 , . . . , xI } and Y = {y1 , . . . , yJ } are sets of corresponding
points in the template and the reference image.
min
z,f
s.t.
Michael Stiglmayr
I P
J
P
2
2
zij kyj − f (xi )k + λ kLf k − ξ
i=1 j=1
PI +1
i=1 zij
PJ+1
j=1 zij
I P
J
P
zij
i=1 j=1
=
=
zij ∈
f ∈
1
1
{0, 1}
F
∀j ∈ {1, . . . , J}
∀i ∈ {1, . . . , I }
∀i ∈ {1, . . . , I +1} ∧ j ∈ {1, . . . , J +1}
Mixed-integer programming for registration
University Erlangen-Nürnberg
Robust Point Matching (RPM)
[Chui and Rangarajan, 2003]
X = {x1 , . . . , xI } and Y = {y1 , . . . , yJ } are sets of corresponding
points in the template and the reference image.
min
z,f
s.t.
Michael Stiglmayr
I P
J
P
2
2
zij kyj − f (xi )k + λ kLf k − ξ
i=1 j=1
PI +1
i=1 zij
PJ+1
j=1 zij
I P
J
P
zij
i=1 j=1
=
=
zij ∈
f ∈
1
1
{0, 1}
F
∀j ∈ {1, . . . , J}
∀i ∈ {1, . . . , I }
∀i ∈ {1, . . . , I +1} ∧ j ∈ {1, . . . , J +1}
Mixed-integer programming for registration
University Erlangen-Nürnberg
Robust Point Matching (RPM)
[Chui and Rangarajan, 2003]
X = {x1 , . . . , xI } and Y = {y1 , . . . , yJ } are sets of corresponding
points in the template and the reference image.
min
z,f
s.t.
Michael Stiglmayr
I P
J
P
2
2
zij kyj − f (xi )k + λ kLf k − ξ
i=1 j=1
PI +1
i=1 zij
PJ+1
j=1 zij
I P
J
P
zij
i=1 j=1
=
=
zij ∈
f ∈
1
1
{0, 1}
F
∀j ∈ {1, . . . , J}
∀i ∈ {1, . . . , I }
∀i ∈ {1, . . . , I +1} ∧ j ∈ {1, . . . , J +1}
Mixed-integer programming for registration
University Erlangen-Nürnberg
Robust Point Matching (RPM)
[Chui and Rangarajan, 2003]
X = {x1 , . . . , xI } and Y = {y1 , . . . , yJ } are sets of corresponding
points in the template and the reference image.
min
z,f
s.t.
Michael Stiglmayr
I P
J
P
2
2
zij kyj − f (xi )k + λ kLf k − ξ
i=1 j=1
PI +1
i=1 zij
PJ+1
j=1 zij
I P
J
P
zij
i=1 j=1
=
=
zij ∈
f ∈
1
1
{0, 1}
F
∀j ∈ {1, . . . , J}
∀i ∈ {1, . . . , I }
∀i ∈ {1, . . . , I +1} ∧ j ∈ {1, . . . , J +1}
Mixed-integer programming for registration
University Erlangen-Nürnberg
solvable to global optimality with
◮
commercial software (CPLEX) – only very small examples
◮
branch & bound w.r.t. assignment variables zij
◮
branch & bound w.r.t. continuous variables f
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
solvable to global optimality with
◮
commercial software (CPLEX) – only very small examples
◮
branch & bound w.r.t. assignment variables zij
◮
branch & bound w.r.t. continuous variables f
local / heuristic approaches
◮
alternate convex search (ACS)
◮
ACS with simulated annealing
◮
genetic algorithms
◮
etc.
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Branch & Bound Algorithm — “divide et impera”
min
f (x, y )
s.t.
(x, y ) ∈ S
x ∈ Zn
x,y
◮
generate subproblems by splitting the feasible region into several
subregions (branching)
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Branch & Bound Algorithm — “divide et impera”
min
f (x, y )
s.t.
(x, y ) ∈ S
x ∈ Zn
x,y
◮
generate subproblems by splitting the feasible region into several
subregions (branching)
◮
for each subproblem calculate a lower bound and a feasible solution
(bounding)
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Branch & Bound Algorithm — “divide et impera”
min
f (x, y )
s.t.
(x, y ) ∈ S
x ∈ Zn
x,y
◮
generate subproblems by splitting the feasible region into several
subregions (branching)
◮
for each subproblem calculate a lower bound and a feasible solution
(bounding)
◮
if the obtained lower bound is larger then a feasible solution of an
other subproblem discard this branch (pruning)
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Branch & Bound Algorithm — “divide et impera”
min
f (x, y )
s.t.
(x, y ) ∈ S
x ∈ Zn
x,y
◮
generate subproblems by splitting the feasible region into several
subregions (branching)
◮
for each subproblem calculate a lower bound and a feasible solution
(bounding)
◮
if the obtained lower bound is larger then a feasible solution of an
other subproblem discard this branch (pruning)
◮
if the lower bound matches the feasible solution of a subproblem an
optimal solution for this subproblem is found, this subproblem is
solved, else branch the subproblem
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Branch & Bound Algorithm — “divide et impera”
min
f (x, y )
s.t.
(x, y ) ∈ S
x ∈ Zn
x,y
◮
generate subproblems by splitting the feasible region into several
subregions (branching)
◮
for each subproblem calculate a lower bound and a feasible solution
(bounding)
◮
if the obtained lower bound is larger then a feasible solution of an
other subproblem discard this branch (pruning)
◮
if the lower bound matches the feasible solution of a subproblem an
optimal solution for this subproblem is found, this subproblem is
solved, else branch the subproblem
◮
stop the algorithm either if all subproblems are solved or pruned or
if the gap between the minimum feasible solution and the minimum
lower bound is below an predefined limit ε
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Local & Global Optimization
Figure: Comparing local (top) and global (bottom) optimization for 2D registration
with twisted starting solution.
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Open Questions
◮
Is the RPM model globally solvable for non-rigid mappings?
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Open Questions
◮
Is the RPM model globally solvable for non-rigid mappings?
◮
How many points have to be selected, finding a tradeoff
between accuracy and computational efficiency / solvability?
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Open Questions
◮
Is the RPM model globally solvable for non-rigid mappings?
◮
How many points have to be selected, finding a tradeoff
between accuracy and computational efficiency / solvability?
◮
Can we obtain an upper bound on the registration error
depending on the segmentation (point selection) error?
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Application to the Registration of High-Speed Video
Sequences of the PE-Segment
Figure: High-Speed Video of the PE-Segment
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Scheme of Artificial Voice Generation
Figure: Lohscheller, J. et al, 2004. Quantitative investigation of the vibration pattern
of the substitute voice generator. IEEE Trans. Biomed. Eng. 51 (8), 1394–1400.
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Registration of Consecutive Video Frames
y
f(x)
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Adapting RPM to the Registration of the PE Segment
◮
special structure of the data: closed contours of ordered points
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Adapting RPM to the Registration of the PE Segment
◮
special structure of the data: closed contours of ordered points
◮
highly non rigid
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Adapting RPM to the Registration of the PE Segment
◮
special structure of the data: closed contours of ordered points
◮
highly non rigid
◮
different type of outliers
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Adapting RPM to the Registration of the PE Segment
◮
special structure of the data: closed contours of ordered points
◮
highly non rigid
◮
different type of outliers
◮
the registration function should model the movement of the
soft tissue in the PE segment
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Adaptive Shape Matching
[Stiglmayr et al., 2006]
min
z,f
I P
J
P
i=1 j=1
s.t.
zij (δdij + (1 − δ)dijN − ξ) + λ kLf k
dij =
PI
kf (xi ) − yj k
dijN = 1 − cos(α)
2
α := ∡(~nf (xi ) , ~nyj )
i=1 zij
≤ B1
∀j ∈ {1, . . . , J}
j=1 zij
≤ B2
∀i ∈ {1, . . . , I }
PJ
zij ∈ {0, 1}
2
∀i ∈ {1, . . . , I } ∧ j ∈ {1, . . . , J}
f ∈ F
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Adaptive Shape Matching
[Stiglmayr et al., 2006]
min
z,f
I P
J
P
i=1 j=1
s.t.
zij (δdij + (1 − δ)dijN − ξ) + λ kLf k
dij =
PI
kf (xi ) − yj k
dijN = 1 − cos(α)
2
α := ∡(~nf (xi ) , ~nyj )
i=1 zij
≤ B1
∀j ∈ {1, . . . , J}
j=1 zij
≤ B2
∀i ∈ {1, . . . , I }
PJ
zij ∈ {0, 1}
2
∀i ∈ {1, . . . , I } ∧ j ∈ {1, . . . , J}
f ∈ F
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Adaptive Shape Matching
[Stiglmayr et al., 2006]
min
z,f
I P
J
P
i=1 j=1
s.t.
zij (δdij + (1 − δ)dijN − ξ) + λ kLf k
dij =
PI
kf (xi ) − yj k
dijN = 1 − cos(α)
2
α := ∡(~nf (xi ) , ~nyj )
i=1 zij
≤ B1
∀j ∈ {1, . . . , J}
j=1 zij
≤ B2
∀i ∈ {1, . . . , I }
PJ
zij ∈ {0, 1}
2
∀i ∈ {1, . . . , I } ∧ j ∈ {1, . . . , J}
f ∈ F
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Adaptive Shape Matching
[Stiglmayr et al., 2006]
min
z,f
I P
J
P
i=1 j=1
s.t.
zij (δdij + (1 − δ)dijN − ξ) + λ kLf k
dij =
PI
kf (xi ) − yj k
dijN = 1 −
2
(f (xi+1 )−f (xi−1 ))T (yj+1 −yj−1 )
kf (xi+1 )−f (xi−1 )kkyj+1 −yj−1 k
i=1 zij
≤ B1
∀j ∈ {1, . . . , J}
j=1 zij
≤ B2
∀i ∈ {1, . . . , I }
PJ
2
zij ∈ {0, 1}
∀i ∈ {1, . . . , I } ∧ j ∈ {1, . . . , J}
f ∈ F
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Adaptive Shape Matching
[Stiglmayr et al., 2006]
min
z,f
I P
J
P
i=1 j=1
s.t.
zij (δdij + (1 − δ)dijN − ξ) + λ kLf k
dij =
PI
kf (xi ) − yj k
dijN = 1 −
2
(f (xi+1 )−f (xi−1 ))T (yj+1 −yj−1 )
kf (xi+1 )−f (xi−1 )kkyj+1 −yj−1 k
i=1 zij
≤ B1
∀j ∈ {1, . . . , J}
j=1 zij
≤ B2
∀i ∈ {1, . . . , I }
PJ
2
zij ∈ {0, 1}
∀i ∈ {1, . . . , I } ∧ j ∈ {1, . . . , J}
f ∈ F
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Properties of This Model
◮
mixed integer problem
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Properties of This Model
◮
mixed integer problem
◮
linear in the assignment variables
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Properties of This Model
◮
mixed integer problem
◮
linear in the assignment variables
◮
totally unimodular with respect to the assignment variables
(if the right hand sides B1 , B2 are integer)
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Properties of This Model
◮
mixed integer problem
◮
linear in the assignment variables
◮
totally unimodular with respect to the assignment variables
(if the right hand sides B1 , B2 are integer)
◮
implementation in Matlab with Alternate Convex Search
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Registration of Consecutive Video Frames 2
y
f(x)
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Registration of Consecutive Video Frames 2
y
f(x)
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Registration of Consecutive Video Frames 2
y
f(x)
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Realistic Movement
ξ = 0.01
different outlier parameter
ξ = 0.003
original
registered
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Inserting Artificial Points
without artificial points
with artificial points
original
registered
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Point Tracing
purpose of point tracing
◮
identification of contour points
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Point Tracing
purpose of point tracing
◮
identification of contour points
◮
measuring the velocity of contour points
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Point Tracing
purpose of point tracing
◮
identification of contour points
◮
measuring the velocity of contour points
◮
reference for the adaption of a
multiple-mass-model
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Point Tracing
purpose of point tracing
◮
identification of contour points
◮
measuring the velocity of contour points
◮
reference for the adaption of a
multiple-mass-model
◮
visualization of the tissue movement during one
oscillation
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
3D - Visualization of the Trajectories
6
time
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Algorithm:Forward – Backward Tracing
1. Choose an oscillation cycle (st_frame and end_frame and
the frame with the smallest opened contour m_frame.
2. Select starting points x1, . . . xn in st_frame.
3. Register every frame from st_frame to m_frame-1 with its
consecutive frame.
4. Trace the selected points x1, . . . xn from st_frame to
m_frame.
5. Register st_frame to end_frame and compute the points
x1’, . . ., xn’ in end_frame which correspond to x1, . . ., xn.
6. Register every frame from end_frame to m_frame+1 with its
prior frame.
7. Trace the points x1’, . . ., xn’ from end_frame backwards to
x1’, . . ., xn’ using the obtained registration functions and
compute upper error bound.
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Algorithm:Forward – Backward Tracing
1. Choose an oscillation cycle (st_frame and end_frame and
the frame with the smallest opened contour m_frame.
2. Select starting points x1, . . . xn in st_frame.
3. Register every frame from st_frame to m_frame-1 with its
consecutive frame.
4. Trace the selected points x1, . . . xn from st_frame to
m_frame.
5. Register st_frame to end_frame and compute the points
x1’, . . ., xn’ in end_frame which correspond to x1, . . ., xn.
6. Register every frame from end_frame to m_frame+1 with its
prior frame.
7. Trace the points x1’, . . ., xn’ from end_frame backwards to
x1’, . . ., xn’ using the obtained registration functions and
compute upper error bound.
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Algorithm:Forward – Backward Tracing
1. Choose an oscillation cycle (st_frame and end_frame and
the frame with the smallest opened contour m_frame.
2. Select starting points x1, . . . xn in st_frame.
3. Register every frame from st_frame to m_frame-1 with its
consecutive frame.
4. Trace the selected points x1, . . . xn from st_frame to
m_frame.
5. Register st_frame to end_frame and compute the points
x1’, . . ., xn’ in end_frame which correspond to x1, . . ., xn.
6. Register every frame from end_frame to m_frame+1 with its
prior frame.
7. Trace the points x1’, . . ., xn’ from end_frame backwards to
x1’, . . ., xn’ using the obtained registration functions and
compute upper error bound.
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Algorithm:Forward – Backward Tracing
1. Choose an oscillation cycle (st_frame and end_frame and
the frame with the smallest opened contour m_frame.
2. Select starting points x1, . . . xn in st_frame.
3. Register every frame from st_frame to m_frame-1 with its
consecutive frame.
4. Trace the selected points x1, . . . xn from st_frame to
m_frame.
5. Register st_frame to end_frame and compute the points
x1’, . . ., xn’ in end_frame which correspond to x1, . . ., xn.
6. Register every frame from end_frame to m_frame+1 with its
prior frame.
7. Trace the points x1’, . . ., xn’ from end_frame backwards to
x1’, . . ., xn’ using the obtained registration functions and
compute upper error bound.
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Algorithm:Forward – Backward Tracing
1. Choose an oscillation cycle (st_frame and end_frame and
the frame with the smallest opened contour m_frame.
2. Select starting points x1, . . . xn in st_frame.
3. Register every frame from st_frame to m_frame-1 with its
consecutive frame.
4. Trace the selected points x1, . . . xn from st_frame to
m_frame.
5. Register st_frame to end_frame and compute the points
x1’, . . ., xn’ in end_frame which correspond to x1, . . ., xn.
6. Register every frame from end_frame to m_frame+1 with its
prior frame.
7. Trace the points x1’, . . ., xn’ from end_frame backwards to
x1’, . . ., xn’ using the obtained registration functions and
compute upper error bound.
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Algorithm:Forward – Backward Tracing
1. Choose an oscillation cycle (st_frame and end_frame and
the frame with the smallest opened contour m_frame.
2. Select starting points x1, . . . xn in st_frame.
3. Register every frame from st_frame to m_frame-1 with its
consecutive frame.
4. Trace the selected points x1, . . . xn from st_frame to
m_frame.
5. Register st_frame to end_frame and compute the points
x1’, . . ., xn’ in end_frame which correspond to x1, . . ., xn.
6. Register every frame from end_frame to m_frame+1 with its
prior frame.
7. Trace the points x1’, . . ., xn’ from end_frame backwards to
x1’, . . ., xn’ using the obtained registration functions and
compute upper error bound.
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Algorithm:Forward – Backward Tracing
1. Choose an oscillation cycle (st_frame and end_frame and
the frame with the smallest opened contour m_frame.
2. Select starting points x1, . . . xn in st_frame.
3. Register every frame from st_frame to m_frame-1 with its
consecutive frame.
4. Trace the selected points x1, . . . xn from st_frame to
m_frame.
5. Register st_frame to end_frame and compute the points
x1’, . . ., xn’ in end_frame which correspond to x1, . . ., xn.
6. Register every frame from end_frame to m_frame+1 with its
prior frame.
7. Trace the points x1’, . . ., xn’ from end_frame backwards to
x1’, . . ., xn’ using the obtained registration functions and
compute upper error bound.
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Advantages Using this Algorithm
1. can handle empty frames (closed PE Segment)
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Advantages Using this Algorithm
1. can handle empty frames (closed PE Segment)
2. prevents error propagation
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Advantages Using this Algorithm
1. can handle empty frames (closed PE Segment)
2. prevents error propagation
3. the error of stitching parts of the trajectory occurs in small
opened frames and is therefore small
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Forward Registration - Backward Registration
Theorem
Let F be a space of isometric (length-preserving
kf (x) − f (y )k = kx − y k), bijective representable functions
(e.g. rigid mappings) furthermore let f ∗ ∈ F be an optimal solution
of Problem (1). Then (f ∗ )−1 is an optimal solution of Problem (2).
min
s.t.
min
s.t.
I P
J
P
I P
J
2
P
zij yj − f (xi ) − ξ
zij
i=1 j=1
PI +1
i=1 zij
PJ+1
j=1 zij
=
=
zij ∈
f ∈
I P
J
P
Michael Stiglmayr
1
1
{0, 1}
F
2
zij g (yj ) − xi − ξ
i=1 j=1
PI +1
i=1 zij
PJ+1
j=1 zij
=
=
zij ∈
g ∈
(1)
i=1 j=1
J
I P
P
zij
(2)
i=1 j=1
1
1
{0, 1}
F
Mixed-integer programming for registration
University Erlangen-Nürnberg
Forward Registration - Backward Registration
Theorem
Let F be a space of isometric (length-preserving
kf (x) − f (y )k = kx − y k), bijective representable functions
(e.g. rigid mappings) furthermore let f ∗ ∈ F be an optimal solution
of Problem (1). Then (f ∗ )−1 is an optimal solution of Problem (2).
min
s.t.
min
s.t.
I P
J
P
I P
J
2
P
zij yj − f (xi ) − ξ
zij
i=1 j=1
PI +1
i=1 zij
PJ+1
j=1 zij
=
=
zij ∈
f ∈
I P
J
P
Michael Stiglmayr
1
1
{0, 1}
F
2
zij g (yj ) − xi − ξ
i=1 j=1
PI +1
i=1 zij
PJ+1
j=1 zij
=
=
zij ∈
g ∈
(1)
i=1 j=1
J
I P
P
zij
(2)
i=1 j=1
1
1
{0, 1}
F
Mixed-integer programming for registration
University Erlangen-Nürnberg
Forward Registration - Backward Registration
Theorem
Let F be a space of isometric (length-preserving
kf (x) − f (y )k = kx − y k), bijective representable functions
(e.g. rigid mappings) furthermore let f ∗ ∈ F be an optimal solution
of Problem (1). Then (f ∗ )−1 is an optimal solution of Problem (2).
min
s.t.
min
s.t.
I P
J
P
I P
J
2
P
zij yj − f (xi ) − ξ
zij
i=1 j=1
PI +1
i=1 zij
PJ+1
j=1 zij
=
=
zij ∈
f ∈
I P
J
P
Michael Stiglmayr
1
1
{0, 1}
F
2
zij g (yj ) − xi − ξ
i=1 j=1
PI +1
i=1 zij
PJ+1
j=1 zij
=
=
zij ∈
g ∈
(1)
i=1 j=1
J
I P
P
zij
(2)
i=1 j=1
1
1
{0, 1}
F
Mixed-integer programming for registration
University Erlangen-Nürnberg
Results of Point Tracing
60
85
55
80
50
45
75
40
70
35
65
30
80
85
90
95
100
105
110
115
120
80
85
90
95
100
105
Figure: 2D visualization of the movement of selected contour points over one
oscillation cycle – from opened starting contour (blue) to opened end contour (red)
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Outlook
challenges and further steps
◮
global optimization
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Outlook
challenges and further steps
◮
global optimization
◮
improving alternate convex search with global information
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Outlook
challenges and further steps
◮
global optimization
◮
improving alternate convex search with global information
◮
comparing and adjusting to a multiple-mass-model
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Outlook
challenges and further steps
◮
global optimization
◮
improving alternate convex search with global information
◮
comparing and adjusting to a multiple-mass-model
◮
extension to 3D datasets (needs new recording device)
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Outlook
challenges and further steps
◮
global optimization
◮
improving alternate convex search with global information
◮
comparing and adjusting to a multiple-mass-model
◮
extension to 3D datasets (needs new recording device)
◮
mulitobjective optimization
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Outlook
challenges and further steps
◮
global optimization
◮
improving alternate convex search with global information
◮
comparing and adjusting to a multiple-mass-model
◮
extension to 3D datasets (needs new recording device)
◮
mulitobjective optimization
◮
medical validation
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg
Literature
Chui, H. and Rangarajan, A. (2003).
A new point matching algorithm for non-rigid registration.
Computer Vision and Image Understanding, 89:114–141.
Pfeuffer, F. (2006)
Registrierung medizinischer Bilddaten auf Basis verallgemeinerter
Zuordnungsprobleme.
Master’s thesis, Universität Erlangen-Nürnberg, Institut für
Angewandte Mathematik, 2006.
Stiglmayr, M., Schwarz, R., Klamroth, K., Leugering, G., and
Lohscheller, J. (2006).
Registration of PE segment contour deformations in digital
high-speed videos.
submitted to Medical Image Analysis.
Stiglmayr, M. (2006)
Personal website
http://www.am.uni-erlangen.de/˜stiglmayr
Michael Stiglmayr
Mixed-integer programming for registration
University Erlangen-Nürnberg