www.nr.no
Practical information
INF 5300
Deformable contours, II
1.
Two double lectures about deformable contours.
2.
The lectures are based on articles, references will
be given during the course.
3.
The transparencies will be available on the course
website.
4.
The transparencies define the exam requirements
An introduction
Lars Aurdal,
Norsk Regnesentral,
9th of February 2005
www.nr.no
Plan, lecture I
Plan, lecture II
1.
What is segmentation?
1.
Matlab demonstration
2.
Motivation, why deformable contours?
2.
Variations of the basic snakes
3.
The basic formalism: snakes as presented by Kass
and Witkin.
a. Topological adaptability
b. 3D
3.
The capture range problem
a. Distance measure based solutions
b. Gradient vector flow field based solutions
www.nr.no
www.nr.no
Matlab demonstration
Matlab demonstration
1.
The matlab scripts that we will use in the following
is available on the course web pages.
2.
Most of it is a direct realization of what we went
through in lecture 1.
1.
The script starts by clearing the workspace and
closing all windows. It then loads a test image.
Uncomment the test image you want to use.
www.nr.no
Matlab demonstration
www.nr.no
Matlab demonstration
% Start at scratch
1.
clear all
close all
% Define external force field
% Read test image and convert to double
It then displays this image, calculates the gradient
information and displays the horizontal and vertical
gradients.
F=imread('circle.tif','tif');
F=double(F);
% F=imread('square.tif','tif');
% F=double(F);
% F=imread('u.tif','tif');
% F=double(F);
www.nr.no
www.nr.no
Matlab demonstration
Matlab demonstration
% Display it
figure
imshow(F,[]);
title('Input image');
% We want to use the negative magnitude of the gradient of this image as
external
% force field so we need sobel masks
s_vert=-fspecial('sobel');
s_horz=s_vert';
% Calculate the gradient information.
F_vert=imfilter(F,s_vert,'replicate');
F_horz=imfilter(F,s_horz,'replicate');
% Lets look at these two images
figure
imshow(F_vert,[])
title('Vertical gradients')
figure
imshow(F_horz,[])
title('Horizontal gradients')
www.nr.no
Matlab demonstration
1.
www.nr.no
Matlab demonstration
%
%
%
%
The next step is to invert this gradient, we also
normalize it to 1.
Now lets calculate the negative magnitude of the gradient.
This will be the external force field. In order to allow for
different input images we normalize the gradient image
to 1
P=sqrt(F_horz.*F_horz+F_vert.*F_vert);
P=P/(max(max(P)));
P=-P;
figure
imshow(P,[])
title('Inverse magnitude of gradient vector')
% Last thing, we need the two spatial derivatives
% of our external force field. Calculate these and
% have a look at them.
P_vert=imfilter(P,s_vert,'replicate');
P_horz=imfilter(P,s_horz,'replicate');
figure
imshow(P_horz,[])
title('X derivative of force field')
figure
imshow(P_vert,[])
title('Y derivative of force field')
www.nr.no
www.nr.no
Matlab demonstration
Matlab demonstration
1.
Then we define the snake. You can vary the
number of control points by setting N to different
values.
2.
You also define an initial position for the snake, you
can make this a circle and also “nudge it a little”.
www.nr.no
Matlab demonstration
%
%
%
%
Matlab demonstration
Now lets define our snake, to begin with lets decide
on some small number of control points (you can change
this to your liking, the rest of the program will adapt
gracefully)
N=20;
% Now we need to give the snake a shape.
% and then "nudge" it a little.
www.nr.no
1.
Then set the parameters for the inner forces.
2.
Also set the constants that define the stiffness
matrix.
Lets make it a circle
x0=60*cos(0:(2*pi/(N)):(2*pi-(2*pi/(N))))+128
y0=-60*sin(0:(2*pi/(N)):(2*pi-(2*pi/(N))))+128
3.
Finally, set λ
x0(2)=x0(2)+30;
y0(2)=y0(2)-20;
www.nr.no
www.nr.no
Matlab demonstration
Matlab demonstration
% Define the weights given to the
two terms in the inner energy
% functional. The values are NOT
arbitrary.
1.
w1=0.000001;
w2=0.01;
Set up the A matrix, here we use the diag function
of matlab.
% Define constants for the
stiffness matrix, do not edit this.
alpha=w2;
beta=-w1-4*w2;
gamma=-2*w1+6*w2;
% Define the step size
lambda=0.2;
www.nr.no
Matlab demonstration
www.nr.no
Matlab demonstration
% Define Stiffness matrix. The code below is just a smart way of doing
% this independently of the number of nodes.
% A =[gamma
%
beta
%
alpha
%
0
%
0
%
0
%
alpha
%
beta
beta
gamma
beta
alpha
0
0
0
alpha
alpha
beta
gamma
beta
alpha
0
0
0
0
alpha
beta
gamma
beta
alpha
0
0
0
0
alpha
beta
gamma
beta
alpha
0
0
0
0
alpha
beta
gamma
beta
alpha
alpha
0
0
0
alpha
beta
gamma
beta
beta;
alpha;
0;
0;
0;
alpha;
beta;
gamma];
A=diag(beta,-N+1)+...
diag(alpha*ones(1,2),-N+2)+...
diag(alpha*ones(1,N-2),-2)+...
diag(beta*ones(1,N-1),-1)+...
diag(gamma*ones(1,N),0)+...
diag(beta*ones(1,N-1),+1)+...
diag(alpha*ones(1,N-2),2)+...
diag(alpha*ones(1,2),N-2)+...
diag(beta,N-1)
www.nr.no
1.
All that remains before takeoff is to initialize x and
y.
2.
We also set a maximum number of iterations
3.
Finally, we prepare an image onto which the
progress of the snake is displayed.
www.nr.no
Matlab demonstration
Matlab demonstration
% Initialise x and y
1.
x=x0';
y=y0';
The loop itself is utterly simple.
% The maximum number of iterations
maxiter=500;
% Display results on top of the input
image
figure
imshow(P,[])
title('Snake position')
hold on
www.nr.no
Matlab demonstration
www.nr.no
Matlab demonstration
% Now loop
iter=0;
while(iter<maxiter)
c=rand(1,3); % Randomly color the snake
%plot(x,y,'*','color',c) % Plot the snake control points
lplot(x,y,c) % Interconnect the nodes
iter=iter+1 % Display the iteration number
x=(inv(A+lambda*eye(N)))*(lambda*xomega*getmatind(P_horz,round(x)+1,round(y)+1));
y=(inv(A+lambda*eye(N)))*(lambda*yomega*getmatind(P_vert,round(x)+1,round(y)+1));
dummy=input(['Press return to continue']);
end
www.nr.no
1.
The first thing you might want to try is to run the
snake with only internal forces.
2.
Let’s focus first on the “tension force”.
3.
Do this by initialising the snake to a circle.
4.
Also set the weight for the external force (omega)
to zero.
5.
What happens?
www.nr.no
Matlab demonstration
Matlab demonstration
1.
Take a look at the rigidity part of the internal forces.
2.
Do this by “nudging” the contour a little.
www.nr.no
Matlab demonstration
www.nr.no
Matlab demonstration
www.nr.no
1.
What happens if you set w2 to some negative
value, that is, you favor contours with many
“spikes”.
2.
Beware: when you try this you might soon get an
error saying that the “index is out of bounds”, this is
just due to the fact that this new scheme might
produce x and y values that are outside the image
domain. Nothing is done to handle this exception
and matlab bails out.
www.nr.no
Matlab demonstration
Matlab demonstration
1.
Turn the weight given to the external forces back
on (omega=1).
1.
Also try this with other images like square.tif and
u.tif.
2.
Run the system again.
2.
3.
How do you avoid oscillations when you are close
to an ideal position?
With u.tif you might observe something strange.
What (and why)?
www.nr.no
Topological adaptability and 3D
www.nr.no
Capture range problems
1.
The problem of topology is the following: The classical snake
is connected, it cannot split to adopt to objects with, for
instance, holes.
2.
This can be a problem in certain situations.
3.
1.
As we have just seen the snake must be initialized
fairly close to the final target in order to get
convergence.
This problem has been addressed by many researchers and
solutions exist. See for instance: Topology Adaptive
Deformable Surfaces for Medical Image Volume
Segmentation, Tim McInerney and Demetri Terzopoulos,
IEEE Transactions on Medical Imaging, vol. 18, no. 10, 1999.
2.
To make a really good initialization we need to
have a very good estimate of the solution before
starting the iterative process of adapting the snake.
4.
This work also deals with an extension to 3D.
3.
5.
In practical life, the topological adaptability of snakes is not
necessarily a big issue! The most frequent applications of
snakes use non-topologically adaptable snakes.
So we can find a good solution if we already know
the solution. That is not very interesting of course.
www.nr.no
www.nr.no
Capture range problems
Capture range problems
1.
The problem stems from the short “range” of the
external forces.
1.
One good way of visualizing this is by looking at the
negative of the gradient of the external force field.
2.
The inverse magnitude of the gradient (or
smoothed gradient) will have significant values only
in the vicinity of the salient edges.
2.
These are the very forces that “pull” on the snake.
3.
The next slide shows this for the circle.
4.
Notice that outside the area in the immediate
vicinity of the circle, these forces are negligible.
3.
This basically forces us to initialize the snake very
close to the target contour.
4.
This problem is also known as the capture range
problem.
www.nr.no
Capture range problems
www.nr.no
Capture range problems
1.
www.nr.no
The very same phenomenon is the reason why you
will not get convergence into concavities, there are
simply no forces to “drag” the snake into the
concavity!
www.nr.no
Capture range problems
Capture range problems
1.
Many authors have suggested different methods for
increasing the capture range of the external
gradient vector field.
2.
We will look at one elegant method suggested by
Xu and Prince in their article: Snakes, Shapes and
Gradient Vector Flow, Chenyang Xu and Jerry
Prince, IEEE Transactions on Image Processing,
vol. 7, no. 3, pages 359-369, 1998.
www.nr.no
Capture range problems
Capture range problems
1.
Xu and Prince observed that smoothing will
increase the capture range, but will not provide
convergence within concavities.
2.
Other methods, for instance those based on
distance maps are even better at increasing
capture range, but the concavity problem remains
the same.
3.
www.nr.no
Xu and Princes solution is based on diffusing the
gradient information.
www.nr.no
1.
Remember: In the classical formulation the EulerLagrange equation was:
2.
The right hand side is the negative gradient of the
external potential field. It is the limited “reach” of
this field that poses us problems.
3.
Xu and Princes method consists in replacing this
vector field with another one.
www.nr.no
Capture range problems
Capture range problems
1.
Xu and Prince define the vector field:
2.
It is g that will be the GVF.
3.
The field g is the field that minimizes the following
functional:
1.
The first term will smooth the data, that is, far from
edges the field will be kept as smooth as possible
by imposing that the spatial derivatives be as small
as possible.
2.
Close to edges (where |∇ f| is large) the field is
forced to resemble the gradient of f itself.
3.
So g is smooth far from edges and nearly equal to
the gradient of f close to edges.
4.
The term µ just defines the weight we give the
different terms in the functional.
www.nr.no
Capture range problems
www.nr.no
Capture range problems
1.
It should come as no surprise that this equation has
a corresponding Euler-Lagrange equation.
2.
We treat u and v as functions of time and solve the
equations in an iterative fashion.
3.
The solution is obviously a numerical one, we use
as set of two iterations, one for u and one for v.
www.nr.no
1.
Returning to the previous image of the “u”-shaped
structure we get driving forces that will provide
convergence into concavities.
www.nr.no
Capture range problems
Capture range problems
www.nr.no
1.
Xu and Prince provide extensive material on this
web address. This is an excellent site for further
exploration of the GVF approach to solving the
capture range problem.
2.
In order to run the GVF examples you must place
all GVF matlab files provided at this address where
matlab will find them.
3.
Matlab example!
www.nr.no