Image Mosaicing
Basic Idea:
When the images are taken by rotating the camera about its optical center, there exists a
perspective projection that transforms one image onto the other image. So given a set of
images, they can be combined by finding the transform that exists between them. This is
done by specifying a set of control points in the two images and then solving for the
transform.
Computing the transformation:
Let P2 be the point in the second image being mapped to the point P1 in the first image.
Let T be the transformation.
Then,
P1 = TP
P1 = (x1 y1 z1)t
P1 = (x2 y2 z2)t
T= (h0 h1 h2
h3 h4 h5
h6 h7 h8)
x2 = (h0*x1+h1*y1+h2)/(h6*x1+h7+h8)
y2 = (h3*x1+h4*y1+h5)/(h6*x1+h7+h8)
ie
(x1)h0 + (y1)h1 + h2 - (x1x2)h6 - (y1x2)h7 - (x2)h8 = 0
(x1)h3 + (y1)h4 + h5 - (x1y2)h6 - (y1y2)h7 - (y2)h8 = 0
Thus for each corresponding pair of points, we get 2 equations in h0-h8. We can thus
solve the equation by specifying 5 correspondences (10 equations) in the two images.
However, in case, some of the correspondences are not accurate they may lead to small
errors in the transformation. So we specify more than 5 correspondences for each pair of
images.
The transformation usually maps a point to a floating point number in the second image.
Hence, binary interpolation is used to obtain the intensity at that point.
Cascading images:
Let I1 be the base image, and let I2 be the image that is to be mosaiced with I1. For more
than 2 images, the correspondence between each image to be put in the mosaic must be
computed with the target image. To do this, the following scheme has been used.
1. For every image for which a correspondence with the target image is known, the
transformation is computed.
Repeat the following until transformations have been found for all images to the target:
2. Choose an image I1 for which the transformation is not yet known.
3. Check to see if there is an image I2 whose transformation is computed and the
correspondence between I1 and I2 is known.
If there is no such image go back to step 2 and start searching again
If such an image is found go to step 4.
4. Find the transformation for warping between I2 and multiply this transformation with
that between I2 and It. Thus the transformation associated with I1 is computed.
Adjusting Contrasts:
To adjust the contrast one strategy that can be used is the following
In the region where overlap between two images starts, have a blending factor to blend
both the images and change this blending factor from 1 to 0.
The resulting intensity is (1-alpha) Ia _ alpha Ib
Alpha varies from 0 to 1 in the region of overlap.
But this is slightly more difficult to enforce and therefore in the case of cascading more
than two images. And it is difficult to find the region of overlap.
The way my program blends the two images is that it computes the average intensity at a
location in the target for pixels of various images that map to this location.
Image Size
The image size can be adjusted by seeing the minimum and the maximum x and y values
that are obtained when the source images are warped to the target image. The difference
in these values gives the X and Y extent which determines the size of the image. The
resulting warp can be translated so that its starting location coincides with 0,0.
First a transform is computed as described above. The first image is copied as it is until
the region of overlap with the second image starts. From there onwards, for each point in
the first image, the point on the second image is computed. The point in the image is
determined using a blending function that takes some contribution from both images
depending on the distance from both images. (change if u are not acutally doing this)
Questions
1) How many control points does it take to get a ``good'' transformation between images?
Ans: The theoretical minimum number of points is 5. However the transform obtained
using only 5 control points is not good. Reasons:
a) The control points are specified by the user manually and may not be accurate. Even if
one of the correspondence is bad, the transform obtained will be bad. Hence it is better to
specify more control points and find a transform that minimizes the error between the set
of control points in one image and the corresponding set of control points in the other
image.
b) Since the images are discrete and not continuous it is not possible to exactly specify
the correspondences. There might not be a pixel in the other image corresponding to a
pixel in one image in the discrete domain.
After running the program many times on different inputs, I would emperically say that
the number of points required for getting a good transform is n (I would say 12-15).
2) How does the algorithm behave at the theoretical minimum of the number of control
points?
If all the 5 control points are accurate and evenly spread over the image then the resulting
transform is good. However since it is difficult to specify exact correspondences, most of
the time the transform obtained is bad. The algorithm then combines pixels that are not
same in the two images and the resulting mosaic is bad.
3) From your experiments how does the accuracy of the control points affect the results?
If the number of control points is large (> 15)and only a few (< 3-4) of them are
inaccurate, then the resulting mosaic is good
On the other extreme, if the number of control points is at the theoretical minimum (5)
and all of them are bad, the resulting mosaic is very bad.
I found that if there are about 10-12 good correspondences and fewer than 4-5 bad ones,
the result is reasonably good.