Pattern Recognition 34 (2001) 587}599
Separation of touching handwritten multi-numeral strings
based on morphological structural features
Donggang Yu*, Hong Yan
School of Electrical and Information Engineering, University of Sydney, NSW 2006, Australia
Received 11 February 1999; accepted 20 December 1999
Abstract
In this paper, we develop a new method to separate touching handwritten multi-numeral strings using morphological
structural features. The touching numeral string is preprocessed with an e$cient algorithm for smoothing, linearization
and detection of structural points of image contours. Based on the analysis of morphological structure in a string, the
region of the left two numerals is determined. In this way, the task of separating a multi-numeral string is reduced into to
process several two-numeral strings. This processing is continued until the touching region found consists of only one
numeral. We have tested our method on image samples taken from the US National Institute of Science and Technology
(NIST) database and the experimental results demonstrate the method is e$cient. 2001 Pattern Recognition Society.
Published by Elsevier Science Ltd. All rights reserved.
Keywords: Touching handwritten numerals; Character recognition; Morphological structural features
1. Introduction
It is di$cult to recognize a handwritten numeral string
directly when it contains several touching characters
[1}5]. Most methods only study how to separate numeral strings which consist of two handwritten numerals
[1}5]. In this paper, we develop an e$cient morphological structural technique to separate touching multinumeral strings which may consist of more than two
numerals. If a string contains more than two numerals,
the region of the left two numerals of this string is
determined "rst. In this way, a separation process of
a string which consists of more than two numerals is
reduced to that of a string which consists of two numerals. After a separation, the separated string is processed in the same method until there is no region left
that contains at least two numerals. In Section 2, we
introduce a preprocessing method [6,7]. In Section 3, we
describe separation algorithm of separating a hand* Corresponding author. Tel.: #61-2-9351-4824; fax: #612-9351-3847.
E-mail addresses: yu@ee.usyd.edu.au (D. Yu),
yan@ee.usyd.edu.au (H. Yan).
written multi-numeral strings based on morphological
analysis. Finally, experiment result and conclusions are
given in Sections 4 and 5.
2. Preprocessing
We represent the direction chain code set of kth contour as
C "+c , c ,2, c ,2, c
, c ,,
I
G
L\ L
(1)
where i is the index of the contour pixels. Suppose that
the starting point is the upper left corner. The di!erence
code is de"ned as: d "c !c . In the smoothed conG
G>
G
tour, "d " of all neighboring points equals 0 or 1 [6,7].
G
We can convert a smoothed contour to a set of lines.
A linearized line consists of ordered pixels. Let its direction chain code set be
+cJL[i] (i"0,2, (nJL!1)),,
J
J
(2)
where ln is the lnth line of a smoothed contour and nJL
J
is the number of points of the lnth line. A linearized
line consists of ordered pixels which have following
0031-3203/01/$20.00 2001 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.
PII: S 0 0 3 1 - 3 2 0 3 ( 0 0 ) 0 0 0 1 6 - 9
588
D. Yu, H. Yan / Pattern Recognition 34 (2001) 587}599
Fig. 1. The pattern models and their character representation of di!erent structural points.
property if
d "cJL[ i]!cJL[ j] (i"0,2, k!1), ( j"0,2, k!1),
GH
J
J
(3)
then "d ")1 [7].
GH
For description and recognition of a shape, we need to
make use of reliable structural and geometrical information. The de"nition of di!erent structural feature points
is based on pattern models shown in Fig. 1. Sixteen
di!erent characters are used to represent these structural
points in Fig. 1. The structural points describe the morphological change along the contour of the binary image.
They show convex or concave segments in di!erent directions and the relation between the contour point and the
background. These structural points can be used to represent morphological structure of a string [7]. For
example, the upper convex point (represented with the
character `a) and the upper valley point (represented
with the character `ma) describe the convex and valley
morphological change along the direction of code 4,
respectively. For the recognition of handwritten numeral,
only the left lateral of its contour contains two convex
points of code 7 after analyzing the lateral structures of
numerals 3 and 7 (having a line in the middle of vertical
part of handwritten numeral 7) [7].
3. The separation algorithm based on morphological
structural features
The overall block diagram of our separation algorithm
is shown in Fig. 2. The original sample strings are
shown in Fig. 3 and their preprocessed results are shown
in Fig. 4. In Fig. 4(d) the size of some characters
is big. These are used to help to see those structural
points of a string. Besides the above preprocessing the
separation algorithm consists of the following main
procedures:
(1) Subdivide a string into four structural regions.
(2) A correction of structural regions.
(3) Find a structural region which only consists of the
left two numerals of a string.
D. Yu, H. Yan / Pattern Recognition 34 (2001) 587}599
589
Fig. 2. Block of diagram of the separation algorithm.
(4) Determine the property of hole contours in the region of two left numerals.
(5) Recognize the leftmost numeral of string.
(6) Correction of the structural region of two left numerals.
(7) Preselect touching point and correct it.
(8) Select all other touching points.
3.1. Subdividing a string into four structural regions
In terms of recognition by humans, the touching points
in a numeral string are located in upper and lower of
a numeral string. Let x and y be the middle x and
KGB
KGB
y coordinates of a string, respectively. In terms of the
topological property of structural points, structural point
590
D. Yu, H. Yan / Pattern Recognition 34 (2001) 587}599
Fig. 3. The original images of eight numeral strings.
`Ra can be used to determine the lower region of an
outer contour of a numeral string. Let x and y repJCB
JCB
resent the x and y coordinates of the left point in the
lower region, respectively. Similarly, the right point in the
lower region is represented with x and y , respectivePCB
PCB
ly. If we analyze the required structural regions, the left
and right points of the lower region need to meet the
following conditions:
1. The y coordinate of the candidate region points (point
`Ra) is greater than y .
KGB
2. If some points `Ra meet the above condition, the
leftmost point `Ra and the rightmost point `Ra of
these points `Ra are the left point and the right point
of the lower region, respectively.
Similarly, the left and right points in the upper region
are represented with x , y and x , y , respectively.
JQS JQS
PQS PQS
We use structural points `a to determine the left and
right points in the upper region based on a similar
method. The region between the two points, (x , y )
JQS JQS
and (x , y ), is considered as the upper region. The
PQS PQS
region between the two points, (x , y ) and (x , y ),
JCB JCB
PCB PCB
is considered as the lower region. The region between the
two points, (x , y ) and (x , y ), is considered as the
JQS JQS
JCB JCB
left lateral region. Based on the above procedure, these
region points of the sample strings in Fig. 4 can be found
and shown in Fig. 5. They are represented by characters
characters `a and `Ra, respectively.
structures of some sample images, twenty morphological
structural models are set up, with the leftmost lateral
numeral being 4 or 9. If the leftmost numeral is numeral
4 or 9, then its series of structural points can belong to
one of twenty models. However, if a series of the leftmost
lateral numeral is one of these models, it is not certain
that it is numeral 4 or 9. This is because there are some
spurious morphological structures. Therefore, we construct some morphological structural parameters to
constrain these spurious structures. For example, one
model of these models is: `aP`FaP`f aP`Fa
P`[aP`saP`RaP`oaP`OaP`$aP`SaP`]aP
`[a P`saP`Ra. The example of Fig. 4(d) belongs to
the above model. Two structural parameters are constructed to constrain these spurious structures. These are
the y and x coordinate di!erence between two points
`Ra in the above model and the left point in the upper
region of the string, respectively. We can see that if the
"rst point `Ra is above the second point `Ra in the
above model and two points `Ra in the above model are
on the left of the left point in the upper region or close to
it, then the leftmost numeral of Fig. 4(d) is numeral 4 or 9.
Similarly, we can construct other structural parameters based on di!erent models.
In terms of similar method we can recognize the leftmost numeral if it is numeral 5.
If the leftmost numeral is numerals 4, 9 or 5 and there
are two points `Ra in their models, then the second point
`Ra should be the left point in the lower region. Such an
example is shown in Fig. 4(d).
3.2. Correction of the left point in the lower region
Based on morphological analysis of numerals, the left
stroke of handwritten numeral 4, 9 or 5 is easily to make
selection of the left point in the lower region incorrect.
Therefore, it is necessary to determine whether the leftmost numeral is numerals 4, 9 or 5. Because the structural points can describe the morphological change of
a shape, we search for a series of structural points from
the left point of the upper region to the right point of the
lower region. Based on the analyses of the morphological
3.3. Structural region consisting of two left numerals
in a string
If a string consists of more than two numerals, it is
di$cult to separate these numerals directly since its
morphological structure is too complicated. A practical
method is to determine a region which consists of two left
numerals of this string. Similar to the structural region of
a string, the structural region of two left numerals is
subdivided by four region points which are called the left
D. Yu, H. Yan / Pattern Recognition 34 (2001) 587}599
591
Fig. 4. The preprocessing results and structural points of sample strings.
and right points of the upper region and the left and right
points of the lower region for two left numerals of
a string. It is clear that x and y coordinates of the left
points in the region of two left numerals are x , y
JQS JQS
and x , y , respectively. The x and y coordinates of
JCB JCB
the right point in the upper region of two left numerals
592
D. Yu, H. Yan / Pattern Recognition 34 (2001) 587}599
Fig. 5. The structural region points and all touching points of sample strings.
are represented by x
and y , respectively. This
PQS
PQS
preselected region point should satisfy the following conditions:
1. The candidate right point in the upper region of two
left numerals is a point `a and its y coordinate is less
than 2y /3.
KGB
D. Yu, H. Yan / Pattern Recognition 34 (2001) 587}599
2. If a point `a satis"es the above condition 1 and is
close to the left point of the upper region, then this
point `a is the preselected right point in the upper
region of two left numerals.
Similarly, the right point in the lower region of two left
numerals can be selected and represented as x
and
PCB
y . The left region is between the upper left point and
PCB
the lower left point. The upper region of two left numerals is between the upper left point and the upper right
point. The lower region of two left numerals is between
the lower left point and the lower right point.
3.4. Properties of hole contour in the region of two left
numerals
The property of a hole contour can be used to describe
the morphological structure of a shape. Generally, there
are six types of hole contour, the upper left hole, the
lower left hole, the upper right hole, the lower right hole,
the left hole and the right hole in the region of two left
numerals. Let xK?V[k], xKGLG[k], yK?V[k] and yKGLG[k] repP
J
S
B
resent the x coordinates of the rightmost point and
leftmost point and y coordinates of the upper most and
lower most point, respectively, where k is the index of a
hole contour in the region of two left numerals. In order
to determine the property of a hole contour, we make use
of a set of parameters to check the geometric position of
a hole contour. They are
x "x !xK?V[k]#x !xKGLG[k],
JSS
JQS
P
JQS
J
(4)
x "x !xK?V[k]#x !xKGLG[k],
PSS
PQS
P
PQS
J
(5)
x "x !xK?V[k]#x !xKGLG[k],
JBB
JCB
P
JCB
J
(6)
x "x !xK?V[k]#x !xKGLG[k],
PBB
PCB
P
PCB
J
(7)
y "y !yK?V[k]#y !yKGLG[k],
JSS
JQS
S
JQS
B
(8)
y "y !yK?V[k]#y !yKGLG[k],
PSS
PQS
S
PQS
B
(9)
y "y !yK?V[k]#y !yKGLG[k],
JBB
JCB
S
JCB
B
(10)
y "y !yK?V[k]#y !yKGLG[k]
PBB
PCB
S
PCB
B
(11)
and let x ""x ", x ""x ", x ""x ", x ""x ",
JS
JSS PS
PSS JB
JBB PB
PBB
y ""y ", y ""y ", y ""y " and y ""y ". Based
JS
JSS PS
PSS JB
JBB
PB
PBB
on Eqs. (5) and (9) an example of calculating parameters
x and y is shown in Fig. 5(c).
PSS
PSS
These feature parameters describe the geometric location of a hole contour in the region of two left numerals.
For example, x is the distance along the x direction
JS
between the upper left region point and a hole contour
in the region of two left numerals. Based on these parameters, we can "nd other feature parameters as follows:
If x )x , then this situation is represented by u "6
JS
PS
J
else u "6. If u "6, that means this hole contour is close
P
J
593
to the upper left region point in a string. Otherwise,
if u "6, this hole contour is close to the upper right
P
point in the region of two left numerals. Similarly, if
x )x , then this situation is represented by d "6 else
JB
PB
J
d "6. If y )y , then d "y /y else d "y /y .
P
JS
JB
JS
JS JB
JS
JB JS
Based on the value of d , we can de"ne three parameters
JS
as follows: if 1*d '0.66, then this situation is repreJS
sented by l "6. That means this hole contour is located
K
at about the middle of y coordinate distance between the
upper left point and the lower left point. If 0.66*d '0
JS
and y (y , then this situation is represented by l "6.
JS
JB
S
If l "6, this hole contour is close to the upper position
S
of y coordinate distance between the the upper left point
and the lower left point. Otherwise, if 0.66*d '0 and
JS
y *y , then this situation is represented by l "6.
JS
JB
B
Similarly, we can describe the distance along the y
direction between a hole contour and the upper right
point or the lower right point in a string. If y )y , then
PS
PB
d "y /y else d "y /y . Based on the value of
PS
PS PB
PS
PB PS
d , we can de"ne three parameters as follows: If
PS
1*d '0.66, then this situation is represented by
PS
r "6. If 0.66*d '0 and y (y , then this situation
K
PS
PS
PB
is represented by r "6. If 0.66*d '0 and y *y ,
S
PS
PS
PB
then this situation is represented by r "6. Based on the
B
above feature data, we can construct three classi"cation
cases. In terms of these classi"cation cases, we can determine the property of a hole contour in the region of two
left numerals.
Case 1: u "6 and d "6. In Case 1, the condition
J
J
u "6, d "6 means that this hole contour is close to
J
J
both the upper left point and lower left point in the
direction of x coordinate. In this case, the property of
a hole contour is only based on the parameter l , l and
S B
l . Therefore, the property of a hole contour can be
K
determined as follows:
(1) If l "6, then the hole contour is the upper left hole
S
contour.
(2) If l "6, then the hole contour is the lower left hole
B
contour.
(3) If l "6, then the hole contour is the left hole conK
tour.
Case 2: u "6 and d "6. Based on the similar analysis
P
P
of Case 1, the result of Case 2 can be described.
Case 3: u "6 and d "6 or u "6 and d "6. This
J
P
P
J
case is more complicated than Cases 1 and 2. In Case 3,
the condition u "6 means a hole contour is close to the
J
upper left point in the direction of x coordinate. However, the condition d "6 means a hole contour is close
P
to the lower left point in the direction of x coordinate.
The case of the conditions u "6 and d "6 is contrary
P
J
to the above case. In these cases, the property of a hole
contour is related to l , l , l , r , r and r . There are nine
S B K S B
K
di!erent cases.
594
D. Yu, H. Yan / Pattern Recognition 34 (2001) 587}599
1. If l "6 and r "6, then the hole contour is the upper
S
S
left hole contour. The conditions l "6 and r "6
S
S
mean that the hole contour is close to the upper
position in the region of two left numerals. Therefore,
the selected responsible parameter should be l "6
S
and the property of this hole contour is the upper left
hole contour.
2. If l "6 and r "6, the analysis and result are similar
B
B
to the above Case 3 (1).
3. If l "6 and r "6, then the hole contour is close to
S
B
the upper position in the region of two left numerals to
detect from the left lateral in the region of two left
numerals and to the lower position in the region of
two left numerals to detect from the right lateral in the
region of two left numerals, respectively. In this case,
assume
d "x #x #x #x .
A
JSS
PSS
JBB
PBB
(12)
The parameter d describes that the hole contour is
A
close to the left lateral or right lateral in the region of
two left numerals in its entirety. Therefore,
(a) If d )0, then the hole contour is close to the left
A
lateral and the responsible parameter l is selected.
S
Therefore, this hole contour is the upper left hole
contour or the lower left hole contour.
(b) If d '0, then the hole contour is close to the right
A
lateral and the responsible parameter r is seB
lected. Therefore, this hole contour is the upper
right hole contour or the lower right hole contour.
Similar to 3(a) and 3(b) of Case 3, we can describe six
other cases based on combination of the parameter
l , l , l , r , r and r . Note that the above region points
S B K S B
K
are in the region of two left numerals. Based on the above
algorithm the property of all hole contours in Fig. 5 can
be detected correctly.
3.5. The recognition of the leftmost numeral
Let n represent the leftmost numeral of a string. We
JQ
can set up some morphological structural models of the
leftmost numeral with some types of structural points,
the property of hole and the chain components and the
structure of a special line in the left region. The order of
structural points is searched for in the left region, from
the point (x , y ) to the point (x , y ). For example,
JQS JQSW
JCB JCB
there are the following morphological structural models
of the leftmost numeral:
E Numeral 3:
(1) It has one order of structural points, `Sa
P`]aP`SaP`]a. This means there are two curves in
the convex direction of code 7 along the left lateral of
a string.
(2) Assume l is the index of a line in which there is
L
a second structural point `Sa, the line i is between the
lines l and l and n
is the number of chain code
L
JCB
A
4 on the line i, then if the left lateral numeral is numeral
3, the condition n '3 is met. Using this condition
A
the numeral 3 can be distinguished from numeral
7 (having a line to overlap on the vertical part of
numeral 7).
E Numeral 5:
(1) It has one order of structural points, SP].
(2) Assume y is the y coordinate of the structural
D
point `Sa and
y
"(y !y )/4#y ,
KKGB
JCB
JQS
JQS
(13)
then y 'y
. This condition and condition 1 mean
D
KKGB
there is one curve in the convex direction of code 7 and
this curve is at the upper of the leftmost lateral.
(3) There is neither the upper left hole nor the lower
left hole.
(4) Assume n is the number of chain code 4 beA
tween point `Sa and the left point of the lower region,
then n '2. This means the lower of the leftmost
A
contains the direction change of code 4.
E Numeral 6 (Normal type):
(1) It has one order of structural points, `FaP`[a
and no any order of point `SaP`[a.
(2) There is the lower left hole.
We can describe other structural models for number
0 (normal type), 0 (having no left hole ), 1, 2 (normal type),
2 (having the lower hole), 4 (normal type), 4 (having no
upper left hole), 6 (having no left lower hole), 7, 7 (having
the line to overlap on the vertical part of numeral 7),
8 (normal type), 8 (only having the lower left hole),
9 (normal type), 9 (having no upper left hole). The recognition algorithm of the leftmost numeral is demonstrated
in Fig. 6 based on our models. First, the number of pairs
of structural points `Sa and `]a can be checked from
l to l in the left lateral region and it is represented by
JQS
JCB
k. If k*2, the leftmost numeral is recognized as numeral
3 or 7 (having a line to overlap on the vertical part of
numeral 7). If k"1, the leftmost numeral is recognized as
numeral 2, 4, 5, 7 (having no line to overlap on the
vertical part of numeral 7) or 9. If k"0, the leftmost
numeral is recognized as numeral 0, 1, 6 or 8. If k"1,
there is one point `Sa. We use y to analyze the y
D
coordinate position of this point `Sa. If it is less
than y
, its position is at the upper of the numeral and
KKGB
the leftmost numeral is recognized as numeral 2 or 7.
Otherwise the leftmost numeral is recognized as numerals 4, 5 or 9. Further, we can recognize each leftmost
numeral based on the patterns of the di!erent leftmost
numeral. The leftmost numerals of sample strings in
Fig. 4 can be recognized correctly in terms of the above
algorithm.
D. Yu, H. Yan / Pattern Recognition 34 (2001) 587}599
595
Fig. 6. Block diagram of the leftmost numeral recognition processing.
Sometimes, it is not necessary to recognize the leftmost
numeral correctly for the separation of a numeral string.
For example, although the numeral 0 (without the left
hole) and the numeral 6 (without the left lower hole) have
the same recognition model, the "rst left structural point
`ma of the numeral string is not selected as a touching
point, because this structural point `ma belongs to the
structural point of the leftmost numeral.
It is necessary to note that if the touching model of two
left numerals in a string is double touching, it is possible
that the recognition result of the leftmost numeral is not
correct. For example, if the leftmost numeral is numeral
5 and the touching model is double touching, then it is
recognized as numeral 9 based on the above algorithm.
After the leftmost numeral of the string in the Fig. 4(f) is
separated, the string of two right numerals belongs to this
case. Here numeral 5 is touched with numeral numeral 0.
Their touching model is double-touching. For any
double-touching string there is the touching hole contour. In this case if the leftmost numeral is recognized as
numeral 9 and the upper left hole contour is the touching
hole contour, then the recognition result of the leftmost
numeral should be corrected as numeral 5. Also, there are
other cases which should be corrected based on their
corresponding correction rules.
3.6. Correction of the preselected structural region points
of two left numerals
Sometimes, the preselected right point of two left numerals are incorrect because of special numerals such as
numerals 4 and 9 (having no upper left hole). Therefore,
sometimes it is necessary to correct the preselected right
point.
For example, if n is numeral 4 or 9 (having no
JQ
upper left hole contour) and there are more than
two points `a in the upper region of two left numerals,
then the new right point in the upper region of two
left numerals is the third point `a in the upper
region of two left numerals. It is necessary to note that
all candidate points `a should meet Condition 1 in
Section 3.3.
We can describe other correction rules based on nine
types of unreasonable morphological structures of
preselected right point in the upper or lower regions
of two left numerals. Based on the procedure described
in Section 3.3 and the corresponding correction rules
the upper right and lower right points in the region
of two left numerals in Fig. 4 can be determined,
represented with character `Ua and `Da, and shown in
Fig. 5.
596
D. Yu, H. Yan / Pattern Recognition 34 (2001) 587}599
3.7. Preselecting a touching point and its correction
In most cases, the touching points are point `ma in the
upper region or point `$a in the lower region. Therefore,
a touching point is preselected in the lower or upper
regions of two left numerals. The preselected touching
point can be found by comparing the x coordinates of
the "rst left structural point `ma in the upper region
with the "rst left structural point `$a in the lower region.
The x coordinates of two compared structural points
are represented with x [0] and x [0], respectively. If
SA
BA
x [0]*x [0], then the "rst left structural point `ma in
SA
BA
the upper region is the preselected touching point. Otherwise, the "rst left structural point `$a in the lower region
is the preselected touching point.
In most cases the above preselected point is reasonable, however sometimes this selection is not correct for
there are a lot of complicated morphological structures.
Therefore, sometimes it is necessary to correct the
preselected touching point based on some morphological
structural conditions. For example if n is numeral 4 and
JQ
the preselected touching point is point `$a, then sometimes the stroke of numeral 4 make the preselected touching point incorrect. In this case, we take the following
correction based on the relative morphological analysis:
1. Search the "rst structural point `Sa from the preselected touching point to the right point of the lower
region of two left numerals.
2. If the structural point `Sa is found in Step 1 and its
x coordinate is less than (x
#x
/3) where
KGB
KGB
x
"x #(x !x )/2, then the new touching
KGB
JCB
JCB
JCB
point is the structural point `Sa instead of preselected
touching point. The string of Fig. 5(d) belongs to this
case. Here the correction result is not good because
the horizontal stroke of numeral 4 is not extended.
Another example is that sometimes the upper stroke of
handwritten numeral 5 makes the preselected touching
point incorrect. In this case the correction rule is as
follows:
1. If the leftmost numeral is numeral 5 and the preselected touching point is in the upper region of two left
numerals, the preselected touching point is changed
with a point `$a in the lower region of two left numerals, which is close to the left of the preselected
touching point and its y coordinate is less than
(2y /3).
KGB
2. Furthermore, if the x coordinate of new selected
touching point based on Step 1 is less than (2y /3),
KGB
then search if there is a point which is that it is the
lower region of two left numerals, its x coordinate is
greater than (2x /3) and less than (4x /3) and its
KGB
KGB
chain code is code 6. If there exists such a point, then
this point is "nally selected touching point.
The sample string in Fig. 5(f) belongs to this case.
Based on Step 1 the preselected touching point is
changed. New touching point is in the lower region of
two left numerals. However, if it does not meet the
condition of Step 2 then the processing of Step 2 is not
taken.
We can describe other correction rules based on eight
types of unreasonable morphological structures of the
preselected touching point. Based on the above procedure the touching points of the sample strings in Fig. 4 can
be selected and shown in Fig. 5. If the touching point is
in the upper region, it is represented with character `ma.
If the touching point is in the lower region it is represented with character `$a.
3.8. Selection of the match touching points
In order to separate the leftmost numeral from two left
numeral strings, we need to "nd other touching points.
(1) Determination of single-touching and double-touching
models: As we analyze the distribution of structural
points in the upper and lower region, we can discover
the morphological di!erence between them and determine the touching model of two left numerals.
That is if all points `ma in the upper region of two left
numerals are below the middle of two left numerals
and all points `$a in the lower region of two left
numerals are over the middle of two left numerals,
then the touching model is double-touching. Otherwise, it is single-touching. Based on this rule, the
regions of two left numerals in Figs. 4(c) and (g) are
double-touching and other regions of two left numerals in Fig. 4 are single-touching respectively.
However, there are some false double-touching situations. In these cases, the responding correction processing is needed. For example, although the region
of two left numerals meets the above rule of double
touching, there is no hole in this region. Therefore in
this case this touching model should be single touching.
(2) Determination of the touching hole contour: If the
touching model of two left numerals is double touching, then there is a touching hole contour in the
region of two left numerals. In general, a touching
hole contour is close to the center of the region of two
left numerals. Based on this basic rule, touching hole
contour of Fig. 5(c) can be determined. However, it is
possible that the selected touching hole contour is
incorrect based on the above basic rule for some
special morphological structures of two left numerals. In these cases the corresponding rules need to
be set up. For example there is a rule as follows:
If n is numeral 0 (having the left hole contour)
JQ
in the region of two left numerals, then this left hole
D. Yu, H. Yan / Pattern Recognition 34 (2001) 587}599
contour cannot be selected as the touching hole contour. Based on this rule, the left hole contour in the
region of two left numerals in Fig. 5(g) is not the
touching hole contour. In this example the lower
right hole contour in the region of two left numerals
is determined as the touching hole contour.
(3) The match touching points: For single touching, there
is a match touching point which corresponds to the
selected touching point on the outer contour of two
left numerals. If the selected touching point is in the
upper region, then the corresponding match touching point is in the lower region or otherwise in the
upper region. If there is a minimum distance between
the selected touching point and a candidate match
touching point, then it is a found match touching
point. This is a basic algorithm of "nding a match
touching point. Except Figs. 5(c) and (g), the touching
model of two left numerals in Fig. 5 is single touching
and their match touching points can be found based
on the above basic rule and represented by character
`oa. Also there are some special structural distribution classi"cations between the selected touching
point and structural points in the region of candidate
points. In these cases, the corresponding processing
procedures are needed.
597
For double touching, there is the second touching
point which corresponds to the selected touching
point. If the selected touching point is in the upper
region of two left numerals, the second touching
point is in the lower region of two left numerals.
Otherwise the second touching point is in the upper
region of two left numerals. The second touching
point is found by the above basic algorithm. Also
there are two match touching points on the touching
hole contour. There is the minimum distance between each selected touching point and its corresponding match touching point. The match touching
points of double touching are represented by characters `Ua and `Da on the corresponding touching
hole contour, respectively. The second touching
point of double touching is represented by character
`oa. For two sample strings of Figs. 5(c) and (g) their
touching points can be found and shown in Figs. 5(c)
and (g) based on the above algorithm.
4. Experimental results
Based on the proposed separation method, the handwritten string can be e$ciently separated from the left to
Fig. 7. The separation results of the numeral strings shown in Fig. 3.
598
D. Yu, H. Yan / Pattern Recognition 34 (2001) 587}599
the right of a numeral string until no touching region
which contains two left numerals can be found. Each step
of separation processing creates two parts: the left numeral and the right part of this numeral string. The
sample images of Figs. 3(a)}(c) consist of two numerals.
Their separation results are shown in Figs. 7(a)}(c). The
sample images of Figs. 3(d)}(g) consist of three numerals.
After the "rst separating the left numeral and the right
part of this string are shown in Figs. 7(d), (f), (h) and (j).
After the second separating the left numeral and the right
part of this string are shown in Figs. 7(e), (g), (i) and (k). In
Fig. 3(h), this sample image contains four handwritten
numerals. The "rst, second and third separation results
are shown in Figs. 7(l), (m) and (n), respectively, based
on our algorithm. All separation result images are
smoothed [6].
In order to develop our algorithm we used 600 sample
images taken from the NIST database to design the
required models in Section 3. The test set consists of 3287
sample images in which each string consists of two numerals and the correct separation rate of the test set is
94.8%. The test set consists of 256 sample images in
which each string consists of three numerals and the
correct separation rate of the test set is 84.7%.
In order to test our algorithm, the separated numerals
are identi"ed using an optimized nearest-neighbor classi"er developed by Yan [8]. Based on this recognition
algorithm, if both the left and right numerals of a string
which consists of two numerals are recognized correctly,
then this string is considered being separated correctly.
The correct separation rate is only 84%, because although separation of a string is correct, not all separated
numerals are recognized correctly by the selected algorithm. Therefore, it is not appropriate to replace the
separation rate with the recognition rate [5].
The results obtained using di!erent separation algorithms for the same testing samples of touching hand-
written numerals are shown in Table 1. Each sample
image of this testing sample set consists of two numerals.
Because other methods cannot be used to separate
a handwritten touching numeral string which consists of
more than two numerals, only experimental results for
our method for this situation is shown. Table 1 demonstrates that the method proposed in this paper is better.
Our algorithm is e$cient because useful morphological structural features are used to identify touching
points of a touching string directly. The isolated numerals do not need to be recognized by a classi"cation
procedure in the process of separation. Therefore, the
correct separation rate is not in#uenced by the isolated
character recognition algorithm. Other algorithms are
related to their recognition algorithms. Theoretically, the
models of morphological structures can describe any
di!erence between two di!erent shapes. Therefore, if we
set up more morphological structural models by using
more samples, then the correct separation rate will be
increased.
5. Conclusion
A new and e$cient method has been proposed to
separate handwritten numeral strings. The number of
numerals of this string may be inde"nite. Based on our
technology, the process of separating a string which consists of more than two numerals is reduced to that of
a string which consists of two numerals. The algorithm
for smoothing, linearization and detection of structural
points of image contours is an important preprocessing
procedure. It makes some analyses of morphological
structures possible. Based on these analyses of morphological structures, the region of two left numerals of
a string is extracted. Therefore, the methods for separating a string which only consists of two touching numerals
Table 1
Results obtained using di!erent separation algorithms with the same test samples of touching handwritten numeral strings. The
rejection and substitution rates are calculated in terms of separated numerals. The number in square brackets are provided in the
reference paper
Technique used to select touching points
Algorithm of this paper
(a string consisting of two numerals)
Algorithm of this paper
(a string consisting of three numerals)
Suters et al.'s algorithm [5]
(a string consisting of two numerals)
Fenrich's algorithm [2]
(a string consisting of two numerals)
Fujisawa et al.'s algorithm [1]
(a string consisting of two numerals)
Zhao et al.'s algorithm [3]
(a string consisting of two numerals)
Rejection rate (%)
Substitution rate (%)
0.0
5.2
0.0
0.3
15.3
10.1
5.3 (53.3)
27.1 (5.8)
0.7 (33.5)
21.9 (7.7)
3.4 (38.3)
13.3 (4.4)
D. Yu, H. Yan / Pattern Recognition 34 (2001) 587}599
can be used to process this extracted region of two left
numerals of a string. Also the described separation algorithm includes detecting the property of hole contour, the
recognition of the left numeral, the determination of the
single- or the double-touching model, the determination
of the touching hole contour, the searching of the touching point or match touching point and some correction
procedures in these processes. The experiment results
demonstrate that our separation method is e$cient. The
correct separation rate is 94.8% for 3287 unseen sample
images in which each sample image consists of two handwritten numerals, and 84.7% for 258 unseen sample
images in which each sample image consists of three
handwritten numerals. These samples are taken from the
NIST database.
References
[1] H. Fujisawa, Y. Nakano, K. Kurino, Segmentation methods
for character recognition: from segmentation to document
structure analysis, Proc. IEEE 80 (7) (1992) 1079}1092.
599
[2] R. Fenrich, Segmentation of automatically located handwritten numeric strings, in: From Pixels to Features 111:
Frontiers in Handwriting Recognition, North-Holland,
Amsterdam, 1992, pp. 47}59.
[3] Z. Zhao, M. Suters, H. Yan, Connected handwritten digit
separation by optimal contour partition, Proceedings of
DICTA-93 Conference on Digital Image Computing
Techniques and Applications, 1993, pp. 786}793.
[4] N.W. Strathy, C.Y. Suen, A. Krzyzak, Segmentation of
handwritten digits using contour features, Proceedings of
the Second International Conference on Document Analysis and Recognition, 1993, pp. 577}580.
[5] M. Suters, H. Yan, Connected handwritten digit separation
using external boundary curvature, J. Electron. Imaging
3 (3) (1994) 251}256.
[6] D. Yu, H. Yan, An e$cient algorithm for smoothing binary
image contours, Proceedings of ICPR'96, Vol. 2, 1996, pp.
403}407.
[7] D. Yu, H. Yan, An e$cient algorithm for smoothing, linearization and detection of structure feature points of binary image contours, Pattern Recognition 30 (1) (1997)
57}69.
[8] H. Yan, Handwritten digit recognition using optimized
prototypes, Pattern Recognition Lett. 15 (1994) 207}211.
About the Author*DONGGANG YU received his diploma in the Department of Automation and Control Engineering from
Northeastern University, Shenyang, China, in 1970. From 1970 he worked in the Department of Electrical Engineering of Dalian
University of Technology, Liaoning, China, where he taught and did research work on Information Processing as an assistant lecturer
and lecturer respectively. He was appointed as associate professor in Electrical Engineering at Dalian University of Technology in 1991.
He is currently working on Image Processing and Recognition as a visiting scholar at The University of Sydney, Australia. His research
interests are in the areas of Image Processing, Pattern Recognition and Biomedical Signal Processing.
About the Author*HONG YAN received his B.E. degree from Nanking Institute of Posts and Telecommunications in 1982, M.S.E.
degree from the University of Michigan in 1984, and Ph.D. degree from Yale University in 1989, all in Electrical Engineering. In 1982
and 1983 he worked on signal detection and estimation as a graduate student and research assistant at Tsinghua University. From 1986
to 1989 he was a research scientist at General Network Corporation, New Haven, CT, USA, where he worked on design and
optimization of computer and telecommunications networks. Since 1989 he has been with the University of Sydney where he is currently
a Professor in Electrical and Information Engineering. His research interests include computer animation, signal and image processing,
pattern recognition, neural and fuzzy algorithms. He is an author or co-author of one book and over 200 technical papers in these areas.
Dr. Yan is a fellow of the Institution of Engineers, Australia (IEAust), a senior member of the Institute of Electrical and Electronic
Engineers (IEEE), and a member of the International Society of Optical Engineers, the International Neural Network Society, and the
International Society for Magnetic Resonance in Medicine.