Region Based Segmentation
Computer Vision © Department of Computing, Imperial College
GZ Yang and DF Gillies ! http://www.doc.ic.ac.uk/~gzy
Segmentation of images is crucial to our understanding of them. Consequently much effort has been
devoted to devising algorithms for this purpose. Since the sixties a variety of techniques have been
proposed and tried for segmenting images by identifying regions of some common property. These
can be classified into two main classes:
•
Merging algorithms: in which neighbouring regions are compared and merged
if they are close enough in some property.
•
Splitting Algorithms: in which large non-uniform regions are broken up into
smaller areas which may be uniform.
There are algorithms which are a combination of splitting and merging. In all cases some uniformity
criterion must be applied to decide if a region should be split, or two regions merged. This criterion
is based on some region property which will be defined by the application, and could be one of many
measurable image attributes such as mean intensity, colour etc. Uniformity criteria can be defined by
setting limits on the measured property, or by using statistical measures, such as standard deviation
or variance.
Segmentation by Histogram
Grey level thresholding is the simplest segmentation process, as many objects or image
regions are characterized by constant reflectivity or light absorption of their surfaces. A
brightness constant or threshold can be determined to segment objects and background.
Thresholding is computationally cheap and fast and it is the oldest segmentation method and
is still widely used in simple application. The definition of the threshold value is usually relied
upon the analysis of picture histogram shape.
Region Merging
Merging must start from a uniform seed region. Some work has been done in discovering a
suitable seed region. One method is to divide the image into 2x2 or 4x4 blocks and check
each one. Another is to divide the image into strips, and then subdivide the strips further. In
the worst case the seed will be a single pixel. Once a seed has been found, its neighbours
are merged until no more neighbouring regions conform to the uniformity criterion. At this
point the region is extracted from the image, and a further seed is used to merge another
region.
There are some drawbacks which must be noted with this approach. The process is inherently
sequential, and if fine detail is required in the segmentation then the computing time will be
long. Moreover, since in most cases the merging of two regions will change the value of the
property being measured, the resulting area will depend on the search strategy employed
among the neighbours, and the seed chosen.
Region Splitting
These algorithms begin from the whole image, and divide it up until each sub region is
uniform. The usual criterion for stopping the splitting process is when the properties of a newly
-1-
split pair do not differ from those of the original region by more than a threshold.
The main problem with this type of algorithm is the difficulty of deciding where to make the
partition. Early algorithms used some regular decomposition methods, and for some classes
these are satisfactory, however, in most cases splitting is used as a first stage of a split/merge
algorithm.
Quadtrees for Region Extraction
An important data structure which is used in split and merge algorithms is the quadtree.
Diagram 4.1 shows a quadtree and its relation to the image. Note that in graphics the
quadtree is used in a region splitting algorithm (Warnock's Algorithm) which breaks a
graphical image down recursively from the root node, which represents the whole image, to
the leaf nodes which each represent a coherent region, which can be rendered without further
hidden line elimination calculations. The same use is made of quadtrees for vision.
Whole Image
Quadrants
IMAGE
QUADTREE
Diagram 4.1: Quadtree Decomposition
Quadtrees impose one type of regular decomposition onto an image. To complete the
segmentation process this must be followed by a merging phase. Thus the problem of finding
adjacent neighbours to a given node has been studied (Diagram 4.2). The problem is one of
tree search and efficient algorithms have been published.
Part of an Image
Current Region
Node being checked
for merging
Diagram 4.2: Splitting and Merging with Quadtrees
-2-
Splitting and Merging
Characteristic of the split and merge method was the algorithm due to Horwitz and Plavidis.
This was based on the use of a segmentation tree, which is normally a quadtree. Each node,
say k, in the tree corresponds to a square region of the image, and has stored with it a
maximum Mk and minimum mk brightness (or other search property) value.
The algorithm starts with a cutset of nodes. This consists of five arrays storing the node
properties: {xk,yk,size,Mk,mk}. Note that only the cutset is stored, not the whole tree. For an
image of resolution NxN pixels it is theoretically possible that each array requires N2 entries,
though, in practical cases, where regions do exist, the requirement is much smaller.
Merging is done by comparing adjacent groups of four nodes with a common parent. If they
obey the criterion they are replaced by their parent node. Similarly splitting is done by
breaking non uniform nodes into the four children nodes at the lower level (Diagram 4.3). The
process is constrained by the tree structure, so, when all the splitting and merging is
complete, the final cutset must be grouped into regions. This is essentially a further merging
process.
Diagram 4.3: A cutset is a set of nodes that covers the whole image
A general algorithm for splitting and merging can be described as following:
1.
2.
3.
4.
Define an initial segmentation into regions, a homogeneity criterion, and a pyramid
data structure.
If an region R in the pyramid data structure is not homogeneous, split it into four
child-regions; if any four regions with the same parent can be merged into a single
homogeneous region, merge them. If no region can be split or merged, go to step (3).
If there are any two adjacent regions Ri, Rj (even if they are in different pyramid
levels or do not have the same parent) that can be merged into a homogeneous
region, merge them.
Merge small regions with the most similar adjacent region if it is necessary to remove
small-size regions.
Quadtree Segmentation Algorithms
Although the Horwitz/Plavidis algorithm is essentially based on a quadtree, the whole tree is
never stored complete. This is because of the memory requirement. For an nxn image, there
would be m levels in the tree where n=2m, and {22(m+1) - 1 }/3 nodes in the tree (why ?). Values
of n can be up to 512 (m=9) meaning the number of nodes is 349,525. Since each node has
at least four pointers (to its children) and one to its parent for efficient processing, the
minimum requirement is about 1.5 MBytes. However, with the introduction of cheap VLSI
-3-
memories, quadtree algorithms have become practical propositions in the eighties.
A naive quadtree algorithm can be constructed by building a quadtree upwards from the
image pixels, storing only the mean value of the subtree pixels in each node. To minimise the
computation, the mean is computed as the tree is built.
1.
2.
3.
Construct a quadtree storing at each node the mean of its four children.
Search from the root of the tree to find the node highest in the tree whose mean is in
the required range.
Use neighbour finding techniques to merge adjacent nodes whose mean is within the
desired range.
Although this gives in essence the quadtree method it requires a few modifications before it
is a practical algorithm.
The Variance/Average Pyramid
The measure of mean value along alone gives us no measure of uniformity which is essential
in practical cases. Instead we need to use some statistical measure, and it turns out that
variance is a convenient, and easy to compute property. Hence at each node in the quadtree
we will store both the mean and the variance of the pixels in the subtree.
If the mean of the pixels in a subtree is represented by:
1
µ' j f(x,y)
n
Then the variance is defined as:
1
1
<' j f(x,y)&µ 2' j f(x,y)2&µ 2
n
n
This method of computing the variance however is computationally expensive, and instead
we need a recursive formulation, similar to the mean computation. The formula is:
1
2
2
2
2
2
<k' <k %<k %<k %<k %µ k1%µ k2%µ k3%µ k4 &µ k
3
4
2
1
4
Where k1, ..., k4 are the four children of k. The proof of this formula can be done by induction.
By using it, the tree can be built without undue overheads. For each node we compute:
•
•
•
Mean - 4 additions 1 shift
mean 2 - 1 multiply
variance - 8 additions + 1 shift
-4-
Region features and merging criteria
Clusters and Histograms
Analysis based on mean and variance makes an important assumption, namely that the
feature upon which segmentation is based is distributed normally. That is, if we construct an
histogram of the property it will have the shape that is modelled by the Gaussian distribution,
where the peak occurs at µ (the mean) and has magnitude a. In the standard form the value
of a is taken to be 1/(2B<)½ which ensures that the integral of the distribution is 1. For
modelling histograms we should compute the area under the histogram and then divide by
(2B<)½ to find the correct normalisation. The parameter < is called the variance (F=<½ is the
standard deviation), and measures the flatness of the distribution. Discrimination between
adjacent areas with differing means and standard deviations can be made according to
Fischer's criterion:
*µ 1&µ 2*
> 8
2
2
<1%<2
where ( is a threshold. In other words, if two regions have good separation in their means,
and low variance, then we can discriminate them. However, if the variance becomes high and
the mean difference is low it is not possible to separate them.
Uniformity
For the best effect, the merging threshold for the mean intensity for two adjacent regions, 8,
should be adjusted depending on the expected uniformity of the merged region. Less uniform
regions will require a lower threshold to prevent under merging. The uniformity is a function
of both intensity mean and variance of the region. A suitable heuristic law for combining both
properties into one is:
uniformity ' 1&</µ 2
Note that the uniformity will be in the range 0 to 1 for cases where the samples are all
positive. The threshold value, 8 decreases with the decrease in uniformity as follows:
8'(1&</µ 2)80
This has the advantage that the user need only supply a single threshold, 80.
-5-