Compact Objects Extraction in Noisy Images
Vladimir Volkov [0000-0001-8552-4775]
Saint-Petersburg State Electrotechnical University (LETI);
Saint-Petersburg University of Airspace Instrumentation, Saint-Petersburg, Russia
vl_volk@mail.ru
Abstract. The aim of the work is to study an adaptive algorithm for detecting
and selecting isolated compact objects on monochrome images obtained by remote surveillance systems. The parameters of the objects of interest are the area
and the compactness coefficient. The adaptive multi-threshold approach is based
on the formation of a set of binary slices that are used for morphological processing, during which the parameters of objects are measured. When combining
the slices, a three-dimensional structure is constructed using the effect of percolation. For the selection of each object, the most suitable slice is selected, on
which the object satisfies the accepted restrictions on area and compactness. The
problems of detecting an object in the form of a disk against a noise background
and distinction such an object from another in the form of a square are considered.
As a decisive statistic, a sampled coefficient of perimeter elongation of the object
is used, which is associated with the optimal selection threshold. The characteristics of detection and discrimination are obtained. The efficiency of the algorithm has been tested on real images containing compact objects. This algorithm
is also applicable to the isolation of bacteria and spores on biological sections, to
detect inhomogeneities in materials and tissues.
Keywords: Multi-threshold image segmentation, Object detection, Selection
and distinction, Percolation.
1
Introduction
The problems of detection, extraction and localization of the objects of interest in noisy
images are relevant in the analysis of images obtained by various remote sensing systems and thus are being intensively studied in the last decades [1–6]. The main differences between objects and noise structures are the connectivity of the object's points,
the isolation of objects from each other, and the contrast of intensities. In fact, the segmentation problem is solving, i.e. the separation of the original image into regions.
Threshold segmentation is most often used, and thresholds can be global or local.
Theoretical threshold values for a given optimality criterion can only be used with
known statistics of objects and background. In practical tasks, such information is not
available, so the adaptation of threshold levels is used. Among the algorithms wellknown Otsu and Bradley-Roth algorithms stand out. These algorithms use the original
2
image to form thresholds, and do not take into account the properties of objects of interest and the results of segmentation in any way. Many objects of interest are characterized by compactness, which can be quantified and used to improve the quality of
detection and selection.
Various variants of multi-threshold processing are based on the properties of the
histogram of the original image [7,8], and as a rule do not take into account the properties of objects of interest and the results of their selection. For heterogeneous objects,
an approach is investigated in [6], which involves the construction of a three-dimensional hierarchical structure of objects based on multi-threshold processing using the
percolation effect.
Considering the prospects of using the selection results to solve the problems of distinguishing and recognizing objects of interest by shape, an important task is to evaluate
the effectiveness of the algorithm under the action of noise. This article examines the
problems of detecting a compact object in the form of a disk against a background of
noise and distinguishing this object from another in the form of a square. As a decisive
statistic, a sample coefficient of perimeter elongation of the object is used, which is
associated with an optimal selection threshold that provides a minimum of this coefficient.
2
Multi-threshold Object Selection Algorithm
2.1
Multi-threshold Object Selection Approach and Parameters of the
Algorithm
The detection and selection of compact objects taking into account the restrictions on
the area and the compactness coefficient are considered in [6]. The investigated multithreshold selection algorithm uses as a useful feature the coefficient of perimeter elon2
gation of the object PS = P /4πS, where P is the perimeter of the object, S is its area
[9]. This characteristic is a geometric invariant and it has a minimum theoretical value
equal to one for a disk-shaped object. However, measured in noisy images, this coefficient can significantly increase even for a compact object due to the appearance of fractal noise structures at its borders, which dramatically increase the perimeter of the object. This significantly affects the quality of selection, especially with small signal-tonoise ratios.
The multi-threshold algorithm proposed and investigated in [6] builds a three-dimensional hierarchical structure of objects based on a set of binary intensity slices obtained with increasing threshold values. In this structure, the object of interest can be
located on several binary layers, depending on its intensity and texture. This state of the
object is determined by the rate of decrease of its area KS = ST+ΔT / ST with an increase
in the threshold from value T to T+ΔT. This coefficient depends on the threshold value
T and reflects properties and texture of the object. If an object quickly loses its area or
breaks into small fragments, then KS is small, and vice versa, values of KS close to unity
indicate a large steepness of the boundaries and stability of the area.
3
The limitation of this coefficient affects the number of new objects that appear in the
place of the original one when its fragmentation increases with an increase in the threshold. An unambiguous determination of the successor of the original object on the next
layer is possible if this coefficient is greater than 0.5. Thus, one of the parameters of
the algorithm is the boundary coefficient of object area stability KP. If for a certain
threshold value (percolation threshold) it turns out that KS < KP, then the original object
is considered “dead”, and new objects are formed from its fragments. When KP is increased to one, this inequality is always satisfied, and new objects are created on each
layer from fragments of the object on the previous layer. Two more parameters of the
algorithm are related to the selection of objects by the minimum area Smin and by the
compactness coefficient PSmax, which limits the PS value of selected objects (see Table
1).
Table 1. Parameters of the adaptive multilevel-based algorithm.
2.2
Name
Symbol
Boundary coefficient of object area stability
KP
KS ≥ KP ; 0.5≤ KP ≤ 1
Restrictions
Minimum area of selected objects
Smin
S ≥ Smin
Maximum value of perimeter elongation
PSmax
PS ≤ PSmax; PSmax ≥ 1
Detection of Objects with Disk Shape
For certainty and clarification of the influence of the shape of the object on the detection
characteristics, consider an object in the form of a disk that appears in Gaussian noise.
In the pixels occupied by the object, there is a (positive) shift in the mathematical expectation of the distribution. The signal-to-noise ratio is defined as a shift related to the
RMS value of the noise. Fig. 1,a shows the input and output images for the optimal
shift detection in every pixel according to the Neyman-Pearson criterion with a false
alarm probability F = 0.01 and with a signal-to-noise ratio d = 2.326. There are two
reasons why it is undesirable to use the accumulation of pixels within the object boundaries to increase the signal-to-noise ratio. Firstly, the size of the object is often unknown, and secondly, the accumulation destroys the boundaries of the object, which
are quite informative.
If you remove small objects in the right image of Fig. 1,a and use a fill, then it is
possible to restore the shape of the object quite accurately. However, with an unknown
background level, there is a problem with setting the optimal threshold. When the signal-to-noise ratio decreases, the ability to restore the shape of the object disappears due
to fragmentation. In the case of Otsu threshold, it is impossible to control the level of
false alarms.
With small signal-to-noise ratios, there are problems with highlighting the shape of
the object. In the case of a local adaptive mean threshold (Bradley-Roth), the problem
of suppressing the background and highlighting the shape of the object remains. This
is illustrated in Fig. 1,b which shows results of segmentation with a signal-to-noise ratio
of d = 2.326.
4
a
b
Fig. 1. Single-threshold binarization: a – input and output images with Neyman-Pearson threshold; b – results of Otsu binarization (left) and Bradley-Roth thresholding (right).
2.3
Object Selection by Area and Compactness
Object selection is an effective means of improving the efficiency of algorithms. Fig.
,a shows the dependences of the probability of a false alarm F on the threshold level T
in the case of removing objects with an area smaller than Smin from the output binary
image.
a
b
Fig. 2. Reducing the probability of a false alarm lgF with threshold T: a – when removing small
objects with S < Smin; b – when additionally removing objects with PS > PSmax
In this case, it is possible to gain in the probability of correct detection of the object by
reducing the threshold level. A similar effect of reducing the probability of a false alarm
F is observed in the case of selection of objects by compactness. Fig. ,b shows the effect
of joint selection of objects by area and compactness at PSmax = 10. Thus, the selection
allows you to significantly clear the output image from the remnants of the background.
2.4
Decisive Statistics and Detection Efficiency
Multi-threshold algorithm determines the best binarization threshold for each object
which corresponds to the minimum value of the compactness coefficient PS. In the case
of pure noise, the selected objects have different compactness coefficients, the distribution of which is shown in Fig. ,a for KP = 0.5. It is fairly well approximated by a
5
lognormal distribution with equivalent mathematical expectation and variance, while
the ratio of the mean to the median is approximately 1.5. When a compact object appears, its value PS is added to previous values.
a
b
Fig. 3. Distributions of the statistics: a – distribution of the compactness coefficient PS over objects in pure noise (LN – lognormal approximation); b – distribution of optimal threshold values
Topt over objects Nobj in pure noise (left) and for disk in noise with d = 10 (right).
a
b
Fig. 4. Detection of the disk in Gaussian noise: a – all extracted objects with d = 10 (left) and
extracted disk by the use of minPS value (right); b – detection probability D depending on signalto-noise ratio d for disk in noise with Smin = 50, PSmax = 100, and false alarm probability F < 0.01.
The informative parameter is the minimum value of PS, which is associated with the
corresponding optimal threshold value Topt. For compact objects, the algorithm finds
minimum of PS at high threshold values, and for background objects – at lower values.
Either minPS or maxTopt can be used as the decisive statistic, but in practice minPS
gives less errors. The generated statistics are used to detect and highlight compact objects among the background ones. Fig. contains output image for all selected objects
(left) and extracted disk by the use of minimum value of PS (right). Detection characteristics were obtained by modelling with number of iterations M = 100 and are presented in Fig. ,b. Selection parameters KP, Smin and PS were chosen so as to obtain false
alarm probability less than F = 0.01 (see Fig. ). Dashed line D0 corresponds to NeymanPearson detector for the same false alarm probability which is calculated by the formulas D0  (d  t NP ) , where tNP = 2.326 is the threshold for F = 0.01, using the probability integral Φ in the Laplace form. As follows from the analysis, the multi-threshold
6
selection algorithm provides some gain in the quality of detection of compact objects
in relation to known procedures.
3
Object Distinction in Noisy Image
3.1
Objects Distinction in Gaussian Noise
Consideration of the problem of distinguishing fully known objects is interesting for
obtaining potential quantitative characteristics of optimal discrimination, for example,
conditional error probabilities or the total error probability [10]. Consider the case of
specifying
real
objects
on
a
discrete
pixel
grid:
i = 1, …, Ny, j = 1, …, Nx, where Nx and Ny determines the size of the entire image.
Let s1 and s2 be precisely known two-dimensional signals with powers P1   s12 (i, j )
i, j
and P2   s22 (i, j ) , moreover, either the signal s1 (hypothesis H1) or the signal s2 (hyi, j
pothesis H2) in an additive mixture with Gaussian noise may appear in the image. The
noise is assumed to be uncorrelated Gaussian, and in each pixel it has zero expectation
and variance σ2. It is required to make a decision in favor of one of the hypotheses by
processing the input image y.
In the case when the object s1 is really present, the observation has the form
y(i, j) = s1(i, j)+ n(i, j), and probability of error (probability of accepting the hypothesis
H2 in the presence of an object s1) is determined by the inequality
 s1 (i, j )s (i, j )   n(i, j )s (i, j ) ( P1  P2 ) / 2 , or
i, j
i, j
 n(i, j ) s (i, j ) ( P1  P2 ) / 2  P1   s1 (i, j )s2 (i, j ) .
i, j
i, j
The left part of the inequality represents a random variable with zero expectation and
variance, and the right part is –PΔ/2. By replacing the variable n = –z/σz, we can obtain
an expression for the error probability p 
 P / 2
  (z)dz , where φ(z) = (2π)
-0.5
exp(–z2/2)

is the Gaussian probability density. Turning to the normalized variable, we obtain a calculation formula using the probability integral in the Laplace form
p = 1 – Φ(α),
(1)
where α = 0.5PΔ/σz = 0.5(PΔ/σ2)0.5. Exactly the same expression is obtained for the second probability of error (the probability of accepting the hypothesis H1 in the presence
of object s2), so the total probability of error is p if the signal images are equally probable.
7
3.2
Optimal Distinction of Objects in the Form of a Disk and a Square
Consider the distinction between two objects of approximately the same area and the
same intensity, one in the form of a disk, the other in a square shape. Fig. shows both
object signals and the difference signal sΔ.
a
b
c
Fig. 5. Object signals (a, b) and the difference signal sΔ (c).
a
b
c
Fig. 6. The results of optimal distinction: a – correlation processing and horizontal sections of
images for hypothesis H1; b – for hypothesis H2; c - theoretical probability of error (PET) and its
estimates (PE) obtained by modeling for the number of iterations M = 100.
Fig. presents the results of correlation processing of s1 and s2 for two hypotheses with
noise variance σ2 = 1 and signal-to-noise ratio d = 4.65 (this ratio represents the relative
shift of expectation in each pixel). Below are the horizontal sections of these images.
The results of modeling optimal distinction process are shown in Fig. ,c for the number of iterations M = 100. In this case, values of d are quite small, but it can be seen that
acceptable error probabilities are achieved when the signal-to-noise ratio for the difference signal should be greater than 10.
3.3
Multi-threshold Algorithm for Object Selection and Distinction by
Geometric Features
The multi-threshold algorithm selects isolated objects on each layer, and selects its own
layer for segmentation of a separate object based on the minimum value of the com-
8
pactness coefficient PS for this layer. In the case of processing a model image containing a disk and a square, the estimates of this coefficient turn out to depend on the signalto-noise ratio d, as shown in Fig. . With sufficiently large signal-to-noise ratios, these
coefficients tend to theoretical values (PS = 1 for a pure disk and PS = 1.2441 for a pure
square). Thus, recognition of each of the objects by the absolute value of the measured
coefficient PS is not possible.
Fig. 7. Dependence of PS on signal-to noise ratio d for the disk and the square (rect).
However, the difference in these estimates allows you to select a disk and a square if
these objects are present in the image together (Fig. ).
a
b
c
Fig. 8. Results of object selection in noise for d = 20; 5; 3 (a; b; c). The color scale corresponds
to Topt values.
Fig. 9. Error probability for adaptive multi-threshold distinction of the disk from the square obtained by modeling for the number of iterations M = 100.
9
Thus, the problem of resolving objects by their compactness is actually solved. In this
case, the algorithm makes it possible to distinguish in noise a disk (the most compact
object) from another object with less compactness. An increase in the noise level leads
to a loss of the shape of objects due to the appearance of fractal noise processes along
the perimeter. The algorithm confidently distinguishes a disk from a square at d > 6,
but at lower values its efficiency drops sharply, as shown in Fig. .
Distinguishing objects by the compactness coefficient solves the problem of the unknown size of the object, however, it is clear from the comparison with optimal processing that there is a significant resource for improving the quality of distinguishing
objects by using additional shape features.
4
Object Extraction in Real Images
Two examples (Fig. and Fig. ) illustrate the effectiveness of using the compactness
coefficient PS to distinguish objects by shape in remote sensing images. Each isolated
object is selected separately, and can be localized and measured.
a
b
Fig. 10. Real image with objects (a) and results of object extraction (b).
a
b
Fig. 11. Real Synthetic Aperture Radar (SAR) image with compact objects (a) and results of
extraction (b).
10
5
Conclusions
The problem of detecting and distinguishing compact objects on monochrome images
generated by remote surveillance systems is considered. An adaptive multi-threshold
algorithm with selection of objects by area and by the coefficient of perimeter elongation
was chosen for the study. The algorithm uses binary slices for morphological processing,
which includes measuring the geometric characteristics of objects and using them to optimize the threshold for each object. The problem of connecting adjacent binary layers
is solved on the basis of percolation and estimation of changes in the area of objects.
The problem of detecting an object in the form of a disk against the background of
Gaussian noise is considered. The use of selection allows you to gain in the detection
probability with pixel-by-pixel threshold processing by reducing the threshold level. At
the same time, the shape of the selected object of interest is preserved.
The problem of optimum distinguishing objects by shape is considered with calculation a probability of error. An example of distinguishing a disk-shaped object from a
square-shaped object is given. The possibility of using the compactness coefficient to
solve the problem of discrimination with unknown sizes of objects is investigated. An
algorithm for multi-threshold selection of objects has been developed, taking into account the limitations of the area and the compactness coefficient. The effectiveness of
the algorithm was tested on real images containing compact objects of interest.
References
1. Cheng, G., Han, J.: A survey on object detection in optical remote sensing images. ISPRS J.
of Photogrammetry and Remote Sensing 117, 11–28 (2016).
2. Arias-Castro, E., Grimmett, G.R.: Cluster detection in networks using percolation. Bernoulli
19(2), 676–719 (2013).
3. Patil, G.P., Taillie, C.: Upper level set scan statistic for detecting arbitrarily shaped hotspots.
Environmental and Ecological Statistics 11, 183–197 (2004).
4. Zhou, W., Troy, A.: An object-oriented approach for analyzing and characterizing urban
landscape at the parcel level. Int. J. of Remote Sensing 29(11) 3119–3135 (2008).
5. Gu, H., Han, Y., Yang, Y., Li, H., Liu, Z., Soergel, U., Blaschke, T., Cui, S.: An efficient
parallel multi-scale segmentation method for remote sensing imagery. Remote Sensing
10(4) 590–608 (2018).
6. Volkov, V.Yu., Bogachev, M.I.: Detection and extraction of objects in digital images. In:
Proc. 9th Mediterranean Conference on Embedded Computing, 9134228 (2020).
7. Shivahare, B.D., Gupta, S.K.: Multilevel thresholding-based image segmentation using
whale optimization algorithm. Int. J. of Innovative Technology and Exploring Engineering
(IJITEE) 8 Issue-12 (2019).
8. Cuevas, E., González, A.: Algorithm based on the behavior of locust swarms. Mathematical
Problems in Engineering Article ID 805357 (2015).
9. Gonzales, R.C., Woods, R.E.: Digital image processing. 4edn. Pearson (2018).
10. Sheikh A.: Principles of transmission and detection of digital signals. In: Digital Communication (2012).