APPLICATION OF MATHEMATICAL MORPHOLOGY OPERATIONS
FOR SIMPLIFICATION AND IMPROVEMENT OF CORRELATION OF IMAGES
IN CLOSE-RANGE PHOTOGRAMMETRY
M. Kowalczyk a, P. Koza a, P. Kupidura a, J. Marciniak b
a
Warsaw University of Technology, Dept. of Geodesy and Cartography, Politechnika Square no. 1, 00-661 Warsaw,
Poland
b
Samsung Electronics Poland
KEY WORDS:
Close Range Photogrammetry, 2-D Feature Extraction, Feature Detection, Industrial Measurement, Digital
Photogrammetry, Correlation, Mathematical Morphology.
ABSTRACT:
Mathematical morphology is a set theory approach, developed by J.Serra and G. Matheron. It provides an approach to digital image
processing based on geometrical shape. Depending on the type of the operation, a specific characteristics of the objects in the image,
like shape, size, neighborhood etc. can be take into an account.Morphology has an ability to make close range photogrammetry
technology faster and to simplify a human work, if it is unavoidable. This paper presents conception of effectively working groups of
morphology functions in particular image cases. Main emphasis is on the repeatability of results, which is important when taking
many photos of one and the same object.Research aims to find right configuration of morphology tools for obtaining exact results in
many photos. That means, that net of features extracted from each photo should fit the nets, taken from others. Our findings are
presented in the paper.
potentially helpful for extracting features in photogrammetry.
Our research is directed towards the right combination of
morphology functions to extract exactly the same set of features
from different views of the same object. The preparation for
feature based matching is examined, because of noise reduction
smoothing functions and surfaces without clearly shown texture
(czegoś mi tu brakuje, bo zdanie nie rozumiem do końca). Other
matching techniques can be applied only after extracting regions
of interest in the images. Those areas are located near extracted
edges.
1. INTRODUCTION
Close range photogrammetry is the technology where each
measured object is imaged on many photos from different
camera positions. This results in huge variety of scale, lighting
and directions of any parts in shown 3D scene. Reconstructing
3D shape of registered scene is connected with finding
similarities on many images. This stage of correlation needs
robust techniques, when automation must be applied.
In photogrammetric process during the work on image pyramids
feature based matching is used at higher pyramids’ levels,
because orientations are unknown or roughly known at the
beginning (Schenk, 1999). When orientation’s approximations
are improved, this technique is followed by least squares
matching or simply cross correlation on higher resolution
pyramids’ levels. It happens usually because there are, beside
mentioned, correlations based only on features in every level of
precision in image space.
The ordinary way of extracting features from images is shown
in figure 1.
Acquisition of the images
Noise removal
First steps are very important, especially in close range, because
of large differences in orientation parameters of taken photos,
thus improvement of reliability in this area has a key role in
modern photogrammetry. Use of digital automation techniques
is possible but produces a lot of troubles. In spite of computers
power’s increase still one of the most important problems to
solve is the huge number of calculations to make in searching
picture area.Morphology like other methods of image filtering
gives potential opportunity to decrease the number of pixels
which are taken to perform image searching in tasks like finding
characteristic marks and image matching. Those are utilized for
any photogrammetric orientations. Other aim of image filtering,
which isthe noise removal, can also be done by morphological
operators.
Extraction of the features
(edges, shapes etc.)
Mathematical
morphology
Vectorisation of elements found
Figure 1. Possibility of use morphological functions in
image recognizing.
Morphology is commonly allocated in remote sensing because
of possibility of image sectioning for objects of different texture
and completing edges. Second application is very promising and
153
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B5. Beijing 2008
which removes any features not similar to geometry of pipe
cracks. In the end after crack enhancement the digital model of
defects is produced.
2. LITERATURE REVIEW
2.1 Basics of Mathematical Morphology
Mathematical morphology is a set theory approach, developed
by J.Serra and G. Matheron. It provides an approach to
processing of digital images that is based on geometrical shape.
Two fundamental morphological operations – erosion and
dilation are based on Minkowski operations. There are two
different types of notations for these operations: Serra/Matheron
notation and Haralick/Sternberg notation. In this paper
Haralick/Sternberg notation, which is probably more often used
in practical applications, is used. In this notation erosion is
defined as follows (Eq. 1) (Serra, 1982):
Architectural monuments as well as industrial objects have
edges and parts which can be possibly detected by usage of
mathematical morphology functions. The author of the paper
(Rodehost, 1997) described dividing images into different parts
where morphology takes role in preparation for final stage in
proposed solution. Opening is taken for separation of stones in
ancient wall.Edge detection and especially discontinuities of
acquired surfaces is the main issue of the paper (Jiang, Bunke,
1999). Instead of photo images authors worked with range
images produced by laser scanner in close range measuration
tasks. Edges were created also by rapid changes of the normal to
the reflected surfaces.
(1)
3. EXPERIMENTS
and dilation (Eq. 2) as:
Research consisted of preparation, noise removal and edge
detection, aimed standard processing of images taken by digital
camera. As pointed at the beginning the main goal was
connected with possibly effective edge extraction.
(2)
where:
3.1 Preparation
In experiment there was implemented common digital camera
Pentax Optio A10 which has got relatively small image analyzer
1/1.8 " (7.18 x 5.32 mm)], producing large amount of noise
even in small iso values. As the industrial object there was
chosen Stereocomparator from Photogrammetric laboratory at
Warsaw University of Technology (Figure 2). Camera was set
to maximum resolution. Unfortunately without possibility of
recording in RAW image format, Jpeg with lowest compression
ratio had to be useed – about 3 MB for 8 megapixel RGB image
(about 8 times compressed).
(3)
B and B̂
are structuring elements and
(4)
Idea of the morphological gradient, operation presented in
(Beucher, 1990), comes out from the mathematical analysis. It
can be defined as following (Eq. 3) (Nieniewski, 1998):
(5)
For the discreet data, where the circle is replaced by the 3x3
pixels square, we can define it as the difference between
dilation and erosion of the input image (Eq. 4) (Nieniewski,
1998):
Figure 2. Digital camera and object utilized in the
experiment
(6)
2.2 Applications
Morphology as a tool for images transformation works in close
range photogrammetry for many years.
It has been used for extracting edges and detecting characteristic
objects in mobile photogrammetry systems to making maps
from images taken from a car, called mobile mapping systems
(Tao, 1998). The author points into robustness of results during
achievement of specific information from series of images as a
main goal for future researching task. Morphology is used
mainly for decrease an area of interest and extracting specific
objects like e.g. road signs. Functions of morphology are also
used in detecting sewer pipes defects. This application demands
preprocessing where authors (Iyer, Sinha, 2005) applied median
filter for noise removal. After that there is developed
combination of functions called linear closing by reconstruction,
Figure 3. Camera stations placement, surrounding
Stereocomparator
Pictures taken with ISO 400 had size in pixels 3264 x 2448,
time of exposition was approximately 1/13 s with aperture 2.80,
not smudgy. There had been chosen 4 from 17 images taken
154
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B5. Beijing 2008
from camera positions surrounding object with about 1 meter
distance (Figure 3).
Example of noisy picture parts is shown in figure 4.
a.)
b.)
c.)
d.)
Figure 4. Random results of brightness registration in images
First action performed in pictures there was reduction of pixel
depth from RGB to gray scale for usage of morphology
algorithms in application Morpho 1.5.
3.2 Noise removal
Then the next step of mostly applied image processing
technologies is noise removal. It can be made by many available
low-pass filters usually working fast and accurately. In
experiment most important parts of images should left unmoved,
thus images had been treated by two basic morphology
functions working oppositely – erosion and dilation or those
composition – opening and closing. Each case needs proper
structural element depending on size of grains in picture noise.
Order of this two functions is important because of existence of
thin dark or light lines in imaged scene. Industrial objects have
usually dark lines which appear on the edge of joining parts of
the device, thus in the beginning should be used erosion or
opening in order to prevent those edges from
disappearing.Morphological filters base on a simple idea of
alternating two operations: opening and closing – removing,
respectively, small high-valued and low-valued objects from the
image. These filters, called generally alternate filters differ from
each other in the type of opening and closing operations applied.
One of the morphological filters seems to be very effective. It is
called Alternate Filter with Multiple Structuring Function
(Element) – the sequence of opening and closing operations
with multiple structuring element. It removes the small objects
from the image but preserves edges and linear objects, what is
very important for the purpose of image filtering as the
preprocessing for edge detection. Its effectiveness was proved
for other types of the images (Kupidura, 2006), (Kupidura,
Koza, 2008) and they can be used in close-range
photogrammetry as well.
The reader is referred to the books and articles of mathematical
morphology for an extended background to morphological
operators (especially (Serra, 1982), (Haralick et al., 1987),
(Nieniewski, 1998) and (Kupidura, 2006)).
In addition common filter like median were checked for
comparison to morphological ones. Results of this research are
presented in figure 5. Local noise level is represented by
absolute difference between quantisation value of taken pixel
and medium value from its surrounding. In graphs are presented
number of pixels from each difference like values in histogram
from image.
Figure 5. Number of standing out pixels from surrounding
neighborhood. X axis shows value of the difference, Y axis
shows relative number of pixels from part of the image
d=2.2 original image a.),
d=1.6 median 3x3 b.),
d=1.4 median 5x5 c.),
d=1.4 opening and closing d.)
As shown in figure 4 combination of morphology functions
reduces noise more strongly then median filter at the same size
of operational element, but there is a danger of removal of
smaller details. That performed action costs much more image
operations. Usually morphological operations eliminate noisy
pixels during other tasks and it is usually a side effect.
3.3 Edge detection
Detection of the edges is the next step leading to extraction of
characteristic features. There are some popular filters which are
usually used to achieve this target. There was taken Sobel and
Previtt filter as the reference to comparison with morphological
tools. As the most effective combination of morphology
functions is taken the difference between two images after
applied erosion and dilation separately. Results are presented in
pictures with inverted brightness (Figure 6 b,c,d).
a.)
b.)
c.)
d.)
Figure 6. Separated edges by filers: Sobel – b.), Prewitt – c.),
morphological gradient – d.), original fragment of the image –
a.)
155
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B5. Beijing 2008
Median filter used before morphological gradient affected
position of the edges, so instead of it should be used
combination of opening and closing.As mentioned at the
beginning the main aim is to find exactly the same set of
features from each photo taken of this object. Therefore feature
should be found ,that depends on the repetitiveness of gained
features like characteristic lines, placed on edges. For better
results should be used adaptive filters [B. Svensson et al., 2006],
which works differently in different image area, and differently
in each image. They cause a lot more calculations in image
processing.
Last part of the process, before acquiring vectors from image, is
thinning of detected linear objects. Therefore the simple
procedure was used , which grouped pixels having enough high
value in line and they were replaced by one in the middle of
each set. That was performed both in horizontal and vertical
direction. This solution was applied after all performed actions
in images, and shows differences connected with noise pixels
and uncompleted lines of edges (Figure 7).
Criteria for choosing right configuration of procedure for
acquiring points from images should be found independent of
illumination conditions as well as setting relevant or flexible
threshold for extracting points.Detected points by opening and
closing for noise removal and morphological gradient for
drawing edges presents figure 9. There are displayed differences
in detection of the same parts of the object.
a.)
b.)
b.)
d.)
Figure 7. Thinned edges after filtering by: ordinary looking for
gradient procedure, threshold=30 – a.), for others threshold=25:
Sobel filter – b.), Prewitt filter – c.), morphological gradient –
d.)
Results suggest applied filters give weights to marked edges,
being in proportion to local gradient, and extracting them by
one threshold leads to noisy detected features near limits of
registration. Figure 8 presents improvement of detection
reliability after applied noise removal filters.
Figure 9. Registration of the points in subsequent images.
4. CONCLUSIONS
a.)
The presented results show the potential of mathematical
morphology in the processing of the close-range
photogrammetry data. The application of the morphological
gradient returns the results that are comparable to the results of
other, widely-used operations, like Sobel and Prewitt operations.
The only significant difference can be seen in the result of the
detection of the edges of the small objects, especially of size of
the noise.
b.)
The noise suppression is the next topic in the preprocessing for
the automatic corellation. The very important issue of filtering
is to preserve the edges of the objects. The morphological Filter
with Multiple Structuring Function seems to be very effective
for this issue.
The effectiveness of morphological operations used for the
preparing of the close-range images for automatic corellations,
especially the effectiveness of the edge preserving of the
different types of filtration and the efficency of the
morphological gradient as the non-directional edge detector are
the subjects of the further research of the authors, so more
precise results of the comparison of morphological and nonmorphological operations will be known in the near future.
c.)
d.)
Figure 8. Found pixels in morphological gradient after applied
noise removal filters: without filtering – a.), median 3x3 – b.),
median 5x5 – c.), opening and closing – d.)
156
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B5. Beijing 2008
Kupidura, P., Koza P., 2008. Radar imagery filtering with use
of the mathematical morphology operations, submitted to print
in Polish Journal of Environmental Science.
ACKNOWLEDGEMENT
Scientific work financed from the science funds for the year
2007-2008 as a research project.
Nieniewski, M., 1998.
Morfologia matematyczna w
przetwarzaniu obrazów, Akademicka Oficyna Wydawnicza PLJ.
Rodehorst, V., 1997. Architectural Image Segmentation
REFERENCES
Using Digital Watersheds. 7th Int. Conf. on Computer Analysis
of Images and Patterns CAIP'97, Kiel, Germany, Sep. 10-12.
Beucher, S., 1990. Segmentation d’Images et Morphologie
Mathématique, PhD thesis, Ecole Nationale Supérieure des
Mines de Paris.
Schenk, T., 1999. Digital Photogrammetry. TerraScience.
Haralick, R. M., Sternberg, S. R. Zhuang, X, 1987. Image
Analysis using Mathematical Morphology, IEEE Transactions
on Pattern Analysis and Machine Intelligence, Ju., 9(4), str.
532-550
Serra, J., 1982. Image Analysis and Mathematical Morphology,
vol.1, Academic Press, London.
Svensson, B., Andersson, M., Smedby, Ö., Knutsson, H., 2006.
Shivprakash Iyer, Sunil K. Sinha J., 2005. Automated condition
assessment of buried sewer pipes based on digital imaging
techniques. Indian Institute of Science, Sep.–Oct., 85, 235–252
Efficient 3-D Adaptive Filtering for Medical Image
Enhancement. Proceedings of the IEEE International
Symposium on Biomedical Imaging. April.
Jiang, X., Bunke, H., 1999. Edge Detection in Range Images
Based on Scan Line Approximation. Computer Vision and
Image Understanding, Vol. 73, No. 2, February, pp. 183–199.
Tao, C.V., 1998. Mobile Mapping Technology for Road
Network Data Acquisition 1. Journal of Geospatial Engineering,
Vol. 2, No.2, pp. 1-13. Tokyo, Japan.
Kupidura, P., 2006. Zastosowanie wybranych operacji
morfologii matematycznej do wydzielania klas pokrycia terenu
na zdjęciach satelitarnych, Phd thesis, Warsaw.
157
The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Vol. XXXVII. Part B5. Beijing 2008
158