For full functionality of ResearchGate it is necessary to enable JavaScript. Here are the instructions how to
enable JavaScript in your web browser.
Conference Paper
A New Algorithm for Greenhouse Corridor Edge Detection
with RGB-D Data.doc

Quan Qiu
o
Conference: IEEE-Cyber 2015, At Shenyang, China
ABSTRACT
This paper presents a new algorithm of corridor edge detection in greenhouse scenes with RGB-D sensor. The
algorithm tries to tackle the detection problem considering corridor’s 3 features, including 3D position, dominant color
and binary edges. Firstly, a height interval will be used to collect 3D points for generating a corridor candidate set.
Secondly, dominant color abstracting will help to delete outliers with similar height but different colors. And connected
region labeling will help to get rid of outliers with similar height and color but different locations. After that dominant
color abstracting and connected region labeling will be repeated on the whole image set, and the resulted region will
have different edges from the one generated from corridor candidate set. Then the common edges will be taken as
root edges. Thirdly, a directional growth strategy will finally mark out the whole corridor edges on canny edges map,
with root edges as its start. Experiments are carried out on real RGB-D data collected in greenhouse, and the
effectiveness of the new algorithm is verified.
Get notified about updates to this publication
Follow publication
Download full-text
Full-text
Available from: Quan Qiu, Sep 30, 2015
SHARE
Page 1
A New Algorithm for Greenhouse Corridor Edge
Detection with RGB-D Data*
Qiu Quan1,2
1. Beijing Research Center of Intelligent Equipment for
Agricultural
Beijing, China
qiuq@nercita.org.cn
Meng Zhijun1,2
2. Beijing Academy of Agricultural and Forestry Sciences
Beijing, China
mengzj@nercita.org.cn
Abstract—This paper presents a new algorithm of corridor
edge detection in greenhouse scenes with RGB-D sensor. The
algorithm tries to tackle the detection problem considering
corridor’s 3 features, including 3D position, dominant color and
binary edges. Firstly, a height interval will be used to collect 3D
points for generating a corridor candidate set. Secondly,
dominant color abstracting will help to delete outliers with
similar height but different colors. And connected region labeling
will help to get rid of outliers with similar height and color but
different locations. After that dominant color abstracting and
connected region labeling will be repeated on the whole image set,
and the resulted region will have different edges from the one
generated from corridor candidate set. Then the common edges
will be taken as root edges. Thirdly, a directional growth strategy
will finally mark out the whole corridor edges on canny edges
map, with root edges as its start. Experiments are carried out on
real RGB-D data collected in greenhouse, and the effectiveness of
the new algorithm is verified.
Keywords—RGB-D; greenhouse corridor detection; dominant
color; connected region; canny;
I. INTRODUCTION
Agricultural robot has been a hot topic in recent decades.
Researchers are paying more and more attentions on it, but
there are still several open questions remain unsolved. One key
problem which constrains the rapid development of agricultural
robot is the environmental perception capability. Agricultural
scenes are unstructured and complex. Detecting, recognizing
and even locating the working targets, such as fruits and weed,
is a tough task for agricultural robots. To solve the problem,
different kinds of sensors have been employed, among which
camera has a definitely high frequency of occurrence [1].
Computer vision can be used to detect the crop rows [2], to
detect the fruits [3-4], to detect weeds [5], and to detect
pests/diseases [6]. Although computer vision has a great
quantity of successful application cases in agriculture, it has an
obvious shortage for obtaining depth information fast and
precisely. As a result, the localization accuracy for working
targets usually can not meet the requirements of operation.
RGB-D sensors can generate both the color image and the
depth image synchronously, leading to a great improvement on
sensing only with cameras. Many researchers observe RGB-D
sensors’ potentiality in the environmental perception. Behken’s
group employs RGB-D data for semantic mapping [7]. Munaro
and Menegatti use RGB-D data to help the service robot
tracking people [8]. Karpathy et al. detect typical indoor
objects with RGB-D sensor [9]. Ren et al. employ RGB-D
sensor to label different kinds of indoor objects [10]. Also,
RGB-D sensor can be used for deformable object tracking [11].
Influenced by its defect in handling blaze conditions, most
researches of RGB-D sensor focus on the indoor scenes. Few
efforts have been made on RGB-D sensor’s application in
outdoor scenes, especially agricultural scenes. In fact,
greenhouse is a semi-outdoor scene and RGB-D sensor can
give good performance most of the time except noon. In this
paper, a new corridor edge detection algorithm based on RGBD data is proposed, which will bring benefit for the
autonomous navigation for agricultural robots on the
greenhouse corridor. The algorithm will determine a group of
pixels as the corridor candidates according to pixels’ height
information. Then, the dominant color of the group will be
found and pixels with dominant color are regarded as having
higher probability to be real corridor pixels. After that, canny
detection and connected region labeling will play important
roles in determining the corridor edge precisely.
II. SEOSOR HEIGHT CALIBRATION
Fig. 1. Installation position of Kinect
Here, Microsoft Kinect is employed as the RGB-D sensor.
To endow the new algorithm with the capability of detecting
This work is supported by National High Technology Research and
Development Program of China Grant #2013AA102406 and National Natural
Science Foundation of China Grant #61305105.
Page 2
corridor candidate pixels, the pre-calibration of the sensor’s
height is essential. Kinect is mounted on a horizontal pedal in
front of the robot with a height as shown in Fig. 1. We define
the global coordination as a left-handed coordinate with
Kinect’s facing direction is the positive Y direction and
Kinect’s left side direction as the positive X direction (Fig. 2).
The height value is calibrated with a supervised solution. The
whole process can be clearly elaborated with the help of Fig.3.
The calibration solution has three steps: first, take a frame of
RGB-D data in an indoor scene with flat floor as shown in Fig.
3(a); second, compute the 3D point cloud with Matlab [12] and
select a set of floor points manually as shown in Fig. 3(b), here
we selected all the points lying in the region of “x∈[0,0.4] & z
∈[-0.2,-0.6]”; Third, compute the selected points’ average
height and take it as the calibration result. After calibration, we
know that a point on the floor will have its z-coordination
around -0.5 meters. In other words, the height of our sensor is
about 0.5 meters.
Fig. 2. Definition of global coordination
a) Color image of the calibration scene
b) Point cloud observed with the same view angle as a)
Fig. 3. Select a guaranteed floor point set manually
III. CORRIDOR EDGE DETECTION
Before starting corridor edge detection, we assume that 3D
point cloud has been obtained. Because Kinect sensor’s depth
range is limited, it is impossible that all the pixels in the color
image will have corresponding depth values. Furthermore,
some pixels holding both color and depth values will greatly
suffer from depth measurement errors. For the purpose of
reducing adverse effects coming from errors, only the pixels
with acceptable depth measurement errors are chosen to form
the 3D point cloud. As a result, our corridor edge detection will
be implemented under the cooperation of all colored pixels and
partial 3D points.
In our algorithm, dominant color abstraction and canny
edge detection are employed as the main tool for the
greenhouse corridor edge detection. Canny algorithm is a very
famous edge detection method in image processing [12]. It can
overcome the influences of noise and carry out pretty good
edge detection results. But canny algorithm just point out all
the edges, leaving the edge recognition problem unsolved. To
solve the problem, the idea of our new algorithm is
determining which edges are the true corridor edges among
plenty of canny edges mainly under the help of the corridor’s
dominant color. Following this idea, the new algorithm can be
divided into two parts. In the first part, dominant color
abstraction will be employed to find the root edges for the
corridor. The dominant color abstraction will run in a corridorcandidate point set generated from 3D point cloud. We call the
corridor-candidate point set as corridor candidate set. Corridor
candidate set is formed by chosen all the points with similar
height values as the sensor height calibration result from 3D
point cloud. As corridor candidate set only contents parts of 3D
points on the corridor, the resulted edges will not be complete
edges and completed ones should be obtained by extending the
resulted edges. So we call them with root edges. Then in the
second part, a directional growth strategy will be applied on the
canny edge map with the root edges as the seeds, and the whole
corridor edges in the color image will be abstracted. The
following contents of this section will give intensive
illuminations for the two parts one by one.
A. Finding Root Edges with Dominant Color Abstraction and
Connected Region Labeling
Following the sensor height calibration result, a height
interval will be determined firstly. 3D points lie in the height
interval will be taken as corridor candidate points. We call the
point set formed by all the candidate points as corridor
candidate set, and call the point set formed by all the pixels in
the corresponding color image as whole image set. The root
edges will be found following the flow chart shown in Fig. 4.
In step 1, dominant color of corridor candidate set is abstracted
based on octree color quantization. Here, our palette has 8
colors. The color which covers most corridor candidate pixels
is determined as the dominant color. Then in step 2, a binary
map which has the same size of the color image is generated.
In the map, a pixel will be assigned to 1 if the color of its
corresponding pixel in the color image is dominant color.
Otherwise, a pixel will be assigned to 0. We name this map as
corridor-candidate-binary map. Here, noise 3D points with
similar height to corridor are removed. In step 3, all the
connected regions of corridor-candidate-binary map will be
labeled. After labeling, the biggest connected region will be
retained and other connected regions will be deleted from
corridor-candidate-binary map. As a result, noise 3D points
with similar height and similar color to corridor are removed.
Page 3
To this step, all the pixels marked with 1 in corridor-candidatebinary map are definitely true corridor pixels. Because the
current connected region is just a part of the whole corridor, the
edges cut it out from the whole corridor region are mistaken as
corridor edges. To get rid of the mistaken edges, step 2 and
step 3 will be repeated on the whole image set. We call the
resulted binary map with dominant-color-binary map. It shows
the biggest connected region after dominant color abstraction
and connected region labeling. Although dominant-colorbinary map will bring in noise pixels with different height
values, it owns all the corridor edges. Then we can find out the
root edges through determining the common edges of corridorcandidate-binary map and dominant-color-binary map.
Fig. 4. Flow chart of finding root edges
B. Abstracting Whole Corridor Edges with Directional
Growth
tolerance = x;
for( all root edge pixels root_edges( i, j ) )
{
for( k = j - tolerance ; k < j + tolerance; k ++ )
{
if( canny_edges_map( i, k ) == 1 )
{
corresponding_edges( i, k ) = 1;
}
}
}
After root edges are determined, our new algorithm
employs them to help for finding whole corridor edges in
(1)
canny edge map. To improve canny edge detection
performances, gray image will be pre-processed by filters,
median filter for example. As a result, canny edges may drift a
few pixels away from the true edge positions. So finding the
corresponding edges of root edges in the canny edge map is the
first step of abstracting the corridor edges. Our new algorithm
uses a simple adjacent area searching method to find the
corresponding edges. The pseudo code of the searching method
is shown in (1). In this search method, we define a small
adjacent area with the help of a tolerance x. For a pixel
root_edge( i, j ) on the root edge, the method will search for
corresponding edge pixels in the interval [ j – tolerance, j +
tolerance ], where j is the column index of root_edge( i, j ).
Then a directional growth method will be launched for
finding out the whole corridor edge. As we assume that the
orientation direction of Kinect is approximately parallel to the
corridor, the corridor will have left edge and right edge. Root
edges or corresponding edges are the middle parts of corridor
edges. So both the left edge and the right edge will have
upward and downward growing process. Taking the upward
growth of the left edge for example, the directional growth
method will be implemented as shown in Fig. 5. ( i, j ) is the
growth start pixel or the newly detected edge pixel. A search in
the eight neighborhood of ( i, j ) will be done. The upward and
leftward growth is fulfilled by two tricks. One is only searching
in the four pixels which are marked with pixel indexes. The
other is searching the four pixels following a fixed order: ( i-1,
j-1 ), ( i-1, j ), ( i-1, j+1 ), ( i, j+1 ). If an edge pixel is found,
the search in ( i, j )’s eight neighborhood will stop and another
new eight neighborhood will start around the new detected
edge pixel. If no edge pixel is found after searching in ( i, j )’s
four marked pixels, the upward and leftward growth will be
stopped. Once all the four growth processes ( left-up, left-down,
right-up, right-down ) are stopped, we can infer that the whole
edges have been found out.
Fig. 5. Upwards growth of left edge
IV. EXPERIMENTS
A. Data collection scene
Experiments are carried out in the greenhouse of Beijing
Academy of Agriculture and Forestry Sciences. All the RGB-D
data are collected under natural light from 4pm to 5pm on
December 25, 2014.
B. 3D point selection and corridor candidate selection rules
As only the pixels with acceptable depth measurement
errors are chosen to form the 3D point cloud, we select the
Page 4
pixels with depth values between 1.2meters and 3.8meters.
When determining corridor candidates, the sensor’s height
calibration result and terrain undulation are all considered. We
use a tolerance of 0.1 meters to handle the terrain undulation.
Then a height interval of [-0.6, -0.4] meters will be employed
to select corridor candidates from 3D point cloud. If a pixel’s z
coordination lies in this interval, we assume it as a corridor
candidate pixel.
C. Performance test of dominant color abstracting and
connected region labeling
Our new algorithm will get rid of outliers from corridor
candidate set through two steps. Outliers can be detected with
the help of two parameters: color and position. In the first step,
outliers with different color from dominant color will be
deleted from candidate set; in the second step, outliers located
out of the main body of corridor will be deleted, and the new
algorithm will take the biggest connected region as the main
body.
Fig. 6. Process of deleting outliers from corridor candidate set
The effectiveness of dominant color abstracting and
connected region labeling is verified through Fig. 6. Fig. 6(a) is
the color image. Fig. 6(b) is the rough corridor candidate set
containing outliers. We can see that many points with similar
color or similar height are mistaken as corridor candidates.
Through dominant color abstracting, pixels with different color
from corridor’s dominant color will be deleted (Fig. 6(c)). But
there are still outliers with similar color in corridor candidate
set. Then all the connected regions will be labeled and only the
biggest connected region is retained (Fig. 6(c)). As a result,
outliers located out of corridor are deleted.
D. Performance test of the new corridor edges abstracting
algorithm
The whole corridor edges abstracting process is revealed by
Fig. 7 through several key links. The first row is color images.
We can see that three frames of RGB-D data are collected with
different sensor orientations. The second row is the result after
dominant color abstracting and connected region labeling are
carried out on the whole image. The biggest connected region
with dominant color is retained. In the third row, the biggest
connected region with dominant color in the corridor candidate
set is found. Then comes the common edges of the above two
biggest connected regions in the fourth row. Canny edge
detection results are shown in the fifth row. Finally, the
corridor edges abstracting results are placed in the last row.
According to the abstracting results, we can infer that the new
algorithm can successfully find right corridor edges in case of
different sensor orientations. But the right corridor edge of the
last frame is incomplete, mainly because of the disconnection
of the right canny edge.
V. CONCLUSIONS
We propose a new greenhouse corridor edge detection
algorithm based on RGB-D sensor data. After pre-processing
of RGB-D data and sensor calibration, the new algorithm can
generate a point set of corridor candidates from the 3D point
cloud. To delete outliers with similar height but different color
as corridor points, dominant color abstracting will be used
firstly. Then connected region labeling will be employed to get
rid of outliers with similar height and color as corridor point by
retaining the biggest connected region. The resulting set is a
sub set of corridor point set. For the purpose of finding root
edges, dominant color abstracting and connected region
labeling will be applied on the whole image domain. The
common edges of the sub set and the dominant color set
generated from whole image will be taken as root edges.
Finally, by finding corresponding edges of root edges in canny
edge map and launching directional edge growing, the whole
corridor edges will be detected. The algorithm is tested on the
RGB-D data collected in a greenhouse at the time of 4 pm to
5pm, December 25th, 2014. Experimental results proved that
the new algorithm can detect corridor edges successfully in
case of different sensor orientations. In future, the algorithm
will be improved to be robust for different lighting conditions.
Also, when canny edges suffer from sudden disconnections, the
new algorithm fails to detect the rest part behind the
disconnection. Additional skills will be developed to overcome
this drawback.
ACKNOWLEDGMENT
The authors would like to thank Jia Shiwei for his kind help
in the experiment implementation.
REFERENCES
[1] J. F. S. Gomes and F. R. Leta. Application of computer vision
techniques in the agricultural and food industry: a review. European
Food Research and Technology, 2012, 235:989-1000.
[2] M. Montalvo, G. Pajares, J. M. Guerrero, J. Romeo, M. Guijarro, A.
Riberro, J. J. Ruz, and J. M. Cruz. Automatic detection of crop rows in
maize fields with high weeds pressure. Expert Systems with
Applications, 2012, 39:11889-11897.
[3] C. Akin, M. Kirci, E. O. Gunes, and Y. Cakir. Detection of the
pomegranate fruits on tree using image processing. First International
Conference on Agro-Geoinformatics, Shanghai, August 2-4, 2012.
[4] M. T. Riquelme, P. Barreiro, M. Ruiz-Altisent, and C. Valero. Olive
classification according to external damage using image analysis.
Journal of Food Engineering, 2008, 87:371–379.
Page 5
Fig. 7. Experimental results of corridor edge detection
Page 6
[5] A. T. Nieuwenhuizen, L. Tang, J. W. Hofstee, and J. Muller. Colour
based detection of volunteer potatoes as weeds in sugar beet fields using
machine vision. Precision Agriculture, 2007, 8(6):267-278.
[6] A. Tellaeche, X. P. Burgos-Artizzu, G. Pajaresc, A. Ribeiro, and C. F.
Quintanilla. A new vision-based approach to differential spraying in
precision agriculture. Computers and Electronic in Agriculture 2008,
60(2):144–155.
[7] J. Stückler, N. Biresev, and S. Behken. Semantic mapping using objectclass segmentation of RGB-D images. IEEE/RSJ International
Conference on Robots and Systems, Vilamoure, Portugal, October,
2012.
[8] M. Munaro and E. Menegatti. Fast RGB-D people tracking for service
robots. Autonomous Robots, 2014, 37(3):227-242.
[9] A. Karpathy, S. Miller, and F. F. Li. Object discovery in 3D scenes via
shape analysis. IEEE International Conference on Robotics and
Automation, Karlsruhe, Germany, May 6-10, 2013.
[10] X. F. Ren, L. F. Bo, and D. Fox. RGB-(D) Scene Labeling: Features and
Algorithms. IEEE International Conference on Computer Vision and
Pattern Recognition, Providence, Rhode Island, USA, June 16-21, 2012.
[11] J. Schulman, A. Lee, J. Ho, and P. Abbeel. Tracking deformable objects
with point clouds. IEEE International Conference on Robotics and
Automation, Karlsruhe, Germany, May 6-10, 2013.
[12] Q. Qiu and W. G. Zheng. Pixel-line based clustering for the 3D point
cloud generated by Kinect depth map. IEEE International Conference
on Information and Automation, Yinchuan, China, August 26-28, 2013.
[13] J. F. Canny. A computational approach to edge detection. IEEE
Transactions on Pattern Analysis and Machine Intelligence, 1986,
8(6):679-698.
Download full-text
View other sources
Hide other sources

A New Algorithm for Greenhouse Corridor Edge Detec...
Available from Quan Qiu · Sep 30, 2015
Data provided are for informational purposes only. Although carefully collected, accuracy cannot be guaranteed. The
impact factor represents a rough estimation of the journal's impact factor and does not reflect the actual current
impact factor. Publisher conditions are provided by RoMEO. Differing provisions from the publisher's actual policy or
licence agreement may be applicable.


REFERENCES (13)
CITED IN (0)

o
o
Sorted by: Order of availability
Order of availability
Appearance in publication
SIMILAR PUBLICATIONS

A simple evolutional model of Habitable Zone around host stars with various mass and low metallicity
Midori Oishi, Hideyuki Kamaya

Energy saving potential of heat insulation solar glass: Key results from laboratory and in-situ testing
Erdem Cuce, Pinar Mert Cuce, Chin-Huai Young

A mental picture of the greenhouse effect
Rasmus E. Benestad
© 2008&dash;2016 researchgate.net. All rights reserved. About us · Contact
us · Careers · Developers · News · Privacy · Terms | Advertising · Recruiting
Join for free
Log in
Email
Password Forgot password?
Keep me logged in
Log in
or log in with
ResearchGate is the professional network for scientists and researchers.
Join for free