The International Symposium on Agricultural and Biosystem Engineering (ISABE) 2013
Real time detection of pin hole on worm-eaten chestnut with 2CCD camera
Soo Hyun Park1
Soo Hee Lee2 Seong Min Kim3 and Sang Ha Noh1,4 *
1
Department of Biosystems & Biomaterials Science and Engineering, Seoul National
University, Seoul, 151-921, Republic of Korea.
2
Life& Technology CO. LTD., Hwaseong-si Gyeonggi-do, 445-964, Republic of Korea.
3
Department of Bioindustrial Machinery Engineering, Chonbuk National University,
Jeonju, 561-756, Republic of Korea.
4
Research Institute for Agriculture and Life Sciences, Seoul National University, Seoul, 151921, Republic of Korea.
* Corresponding Author Tel: +82-2-880-4603, Email : noh@snu.ac.kr
Abstract
Overall quality of chestnut is determined mainly by size, weight, shape and internal
disorders such as decay, worm-eaten, etc. In Korea chestnuts having internal disorders are
picked out manually on individual basis by surface color and pin holes made by worm. The
ultimate purpose of this study is to develop a chestnut sorter using multi-channel vision
system which can detect pin holes and the color of chestnut. Primary, this study is focused to
detection of pin holes on chestnut surface. Both color image and gray-scale image in near
infrared band were captured simultaneously from the chestnut sample with an image
acquisition system constructed with a 2CCD camera and an algorithm to detect pin holes
were developed with binary image processing of the gray scale images. A sorting system
which is consisted of feeder, roller conveyor, a multi-channel camera and automatic
discharging device was developed and used to evaluate the effectiveness of the pin-hole
detection and sorting algorithm. Finally, pin-hole detection and sorting rate were evaluated
great than 85 %, showing possibility of commercialization of the developed system with
further study on analysis of surface color related to internal disorder.
Keywords: Chestnut, Internal disorder, Machine vision, NIR image, On-line detection.
The International Symposium on Agricultural and Biosystem Engineering (ISABE) 2013
Introduction
Chestnut (Castanea spp.) is popular for its abundant nutrients and delicious taste in
Asia and Europe (6). More than 2,000,000 ton chestnuts were produced annually, and the
consumption and production of chestnut continue to rise all over the world according to the
Food and Agriculture Organization Statistics (3). Generally, chestnut quality is measured not
only by internal disorders such as decay, worm-eaten, etc., which are important for consumer
acceptance, but also by external factors such as color, shape, size, and surface status. Most
importantly, the external appearance usually is not altered, at least initially, but it may cause
the moisture loss inside, which leads internal disorders usually accelerating the anatomical
and physiological changes within the tissue (1). On that account, pin holes made by worms
on chestnut hull are of great concern to the chestnut producers and consumers because they
indicate inside of the chestnut is disordered and appearance of caterpillar in peeling process
can make customers feel revolting. As time elapses, pin-hole chestnuts start to decay easily
and eventually infect other sound chestnuts. If chestnut fleshes are decayed, relatively large
dark color is exposed on the hull. In chestnut packing house, sizing of chestnut is usually
done by a hole size of drum but pin-hole and dark color chestnuts are picked up manually on
individual basis. Screening work is labor intensive and its efficiency is not so high since pin
holes are hard to find because their sizes are small and they are located randomly, and dark
color is not apparent.
In order to improve the detecting precision of the defective chestnuts with a
commercialized sorter, a technique based on near-infrared (NIR) spectroscopy or machine
vision have been proposed (4). NIR spectroscopy is a non-destructive measuring method with
the advantages of minimized preparation of samples, fast, cheap, and easy to operate and
environment-friendly (8). It has been conducted to do both quantity and quality analysis of
agricultural products. Liu et al. used NIR spectra in 833 – 2500 nm from chestnuts to detect
moldy chestnut, and it performed the detection ratio over 92.8% (5). Machine vision systems
have been used for evaluation of color grading, detection of surface defects, sizing and shape
detection of fruits and vegetables since the late 1970s (4). Machine vision which uses
cameras instead of human eyes in carrying out measurement and judgment, and it is now
widely applied to commercialized fruits sorters. Fang et al. designed a machine vision for
real-time chestnut rating system (2) and Wang et al. proposed a recognition method of wormeaten chestnuts based on machine vision technique (7).
The ultimate purpose of this study is in developing an on-line chestnut sorting system
consisted of feeder, conveyor, machine vision, automatic discharger, etc., which can be used
at chestnut packing house. However, the aim of present study is focused to construct a multichannel machine vision system, to develop an algorithm to detect pin holes on chestnut hull
and to evaluate performance of the algorithm on-line status.
Materials and methods
Samples
Chestnut samples (Daebo species) were obtained from on-line market, which were
harvested in October 2010 at Kong-ju, the southern area of Korea. The samples for the
experiment include 20 normal fresh chestnuts and. 6 chestnuts having pin-holes on hull.
Image acquisition
Primarily, a CCD camera (Model: KF-F2A, Hitachi, Japan) equipped with a filter
wheel to the camera lens was used to obtain images of normal and defective chestnuts at 8
different wave bands of 720, 740, 780, 840, 880, 900, 940, and 960nm. Those images were
analyzed in view of detecting possibility of pin holes at different wavelength bands.
Secondarily, a 2CCD camera (JAI AD-080GE) which is designed as shown in Fig. 1 was
The International Symposium on Agricultural and Biosystem Engineering (ISABE) 2013
used to capture both the color and NIR images at the same time from a chestnut sample. This
camera uses two prisms in the optical path in order to divide one image to two images and
band-pass filters are installed on each spectral axis to capture color image through a Bayer
Filter Array on one sensor and NIR image in the range of 750 to 900 nm on the other. The
camera has a resolution of 1024 x 768 pixels and is able to capture 30 frames per second in
full frame mode. SDK (Software Development Kit) was provided that we programmed to
allow simultaneous RGB and NIR image acquisition.
Figure 1. Schematic of a 2CCD camera
Lights source
Krypton lamp was used for lighting, and was investigated the spectral characteristic
by wavelength using a spectrometer (USB4000, Ocean Optics, USA). It is determined that
lighting is effective in range between 630nm and 960nm shown as Fig. 2.
Figure 2. Spectral characteristic of a krypton light source
On-line sorting system
A laboratory scale of on-line sorting system which is consisted with feeder, roller
conveyor, image acquisition and discharging units was constructed to evaluate the
performance of pin-hole detection and sorting algorithm developed in this study. A linear
feeder (Fig 3-A) was designed and fabricated for dispersing and aligning the chestnuts
The International Symposium on Agricultural and Biosystem Engineering (ISABE) 2013
supplied in bulk status. The chestnuts fed by linear feeder moves to roller conveyor shown in
Fig. 3-B, which was designed so that each chestnut could evolve slowly on the roller and
moves to image acquisition unit.
Figure 3. Linear feeder (A) for aligning the chestnuts and roller conveyor (B) for rotating and
transferring chestnuts
The camera unit was installed near the discharging end of the roller conveyor and
guide strips consisted with 8 channels was put on the surface of roller under the camera view
area shown as Fig. 4. The role of guide strips is to keep chestnuts aligned in each line until
they drop to discharge actuator. While chestnut samples pass the guide strips, they should
evolve at least one turn so that total surface is exposed to the camera. The camera shutter
operates three times while the sample passes through the camera view window. As the result,
three image frames are taken from each chestnut sample at least and those were analyzed for
detection of pin holes.
Figure 4. Outlook of two multi-channel cameras, guide lines, and discharging area
Method for discharging of defective chestnuts
An automatic discharging unit shown in Fig. 5, which was made with 8 flat
rectangular bars and 8 solenoid actuators, was installed at the end part of the roller conveyor
right after the guide strips. Each bar accounts for each guide channel. If a chestnut sample is
decided to be normal or defective by the image analysis, a signal is transferred to the
corresponding discharge solenoid. Whenever pin-hole samples are sliding on the discharging
The International Symposium on Agricultural and Biosystem Engineering (ISABE) 2013
bars, the corresponding solenoids are actuated and the bar moves up so that flow directions
are changed (Fig. 5-A). Otherwise, the actuators are not on and the bars stay down (Fig. 5-B).
In other words, the defective chestnuts are discharged upward and the normal fall through the
ordinary path. In actuating the discharging bar, the delay time of the signal transfer from the
camera and to actuator reaction is very important in order to synchronize the sliding speed of
chestnut sample on to the bar and the timing of the bar actuation.
Figure 5. Bar-type discharging device actuated by solenoid (when the defective chestnut
passes, it moves up (A) to change direction of the defective, otherwise the bar stays down (B)
Development of pin hole detection algorithm
Detection software for screening worm-eaten and decayed chestnut was developed
using an image acquisition system equipped with a 2CCD camera. Software provided by the
frame grabber SDK was used to be operated on Windows XP configuration and binary image
process was applied to the NIR band images for screening pin holes. Three threshold vales of
90, 100, and 110 were examined to get binary image and the ratios of the major and minor
axes of the blobs in the binary image were computed to reconize if it is a pin hole or not.
Effect of roller conveyor speed (0.15, 0.20 and 0.25 m/sec) on pin hole detection was also
examined.
Results and discussion
Features of Gray-scale image in NIR band
Images captured at different wave bands (Fig. 6) indicate that pin hole on chestnut
surface is clearly visualized in the wavelength range of 740 to 960 nm but it was not in
visible range. Therefore, it was concluded that chestnut images taken in the range of 750 to
950 nm (named as NIR band image in this study) are very useful in detecting pin holes.
Pictures are not shown here but relative brightness of the pin hole was affected by the
lightening method and relative position of the pin hole to the lightening, showing bad effect
of specular reflection. It is remarked that location of pin hole is very random and that surface
color of chestnut becomes locally darker than the normal when chestnut flesh is rotted to
certain degree. Pixel values of this dark area are similar to those of pin hole and affects binary
image processing for pin hole detection. In order to avoid this type of problem,
morphological feature of the dark parts such as ratio of major and minor axes was adopted in
binary image processing as described in pin-hole detection algorithm.
According to the preliminary results on the image characteristics of the pin hole and
decayed chestnut, it was concluded that a 2CCD camera as shown in Fig. 1 would be useful
The International Symposium on Agricultural and Biosystem Engineering (ISABE) 2013
for sorting the decayed and worm eaten chestnuts. In this research, further study is planned to
find if those darker parts could be detected from the color image processing and related to the
decay degree of the chestnut flesh.
Figure 6. Filter images of pin hole on worm eaten chestnut
Features of the images taken by the 2CCD camera
Figure 7 shows the chestnut images captured by the 2CCD camera while chestnuts
were being conveyed in between the guide plates on roller conveyor. It was observed that the
color and gray scale images generated by the both image sensors are well synchronized in
terms of image size and location. Therefore, the mask generated from NIR band image can be
applied to the color image and point processing of two images such as their subtraction and
multiplication can be taken if needed.
Real time detection and separation rate of pin-hole chestnut
In order to simplify the detection algorithms, binary processing and ratio of the major
and minor axes of the blobs in the resulted binary image were adopted to the NIR band
images for detection of pin holes. It was considered as pin hole if the ratio of blob is greater
than 0.3. Effects of threshold value and conveyor speed on detection rate were examined and
results are shown in Table 1 and 2. Detection accuracy of the sound is from 83 to 94% and
that of pin-hole chestnut is 73 to 90% under the test conditions in this study. As the threshold
value increases, adverse effect is given to the sound but favorable effect to the pin-hole
samples. And the test results indicate that about 15% of the sound is discharged to the
defective and about 15 to 20% of the defective to the normal, and the secondary manual
sorting should be accompanied. If we think about absolute amount of the sound and the
The International Symposium on Agricultural and Biosystem Engineering (ISABE) 2013
defective in chestnut production, separation rate of the defective should be maximized in
order to minimize the secondary separation work with the discarded.
Figure 7. Real time color and NIR band images captured from the chestnuts which are
being revolved and transferred on roller conveyer.
Table 1. Detection and separation rate of pin-hole chestnut depending on threshold value
Threshold value
Sound chestnut (%)
Wormhole chestnut (%)
TP: 92
FP: 81
TN: 8
FN: 19
TP: 89
FP: 84
100
TN: 11
FN: 16
TP: 83
FP: 90
110
TN: 17
FN: 10
TP: 88
FP: 85
Total detection rate
TN: 12
FN: 15
Note: True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN)
90
Table 2. Detection and separation rate of pin-hole chestnut depending on roller speed
Roller transfer speed
Sound chestnut (%)
Wormhole chestnut (%)
TP: 89
FP: 84
TN: 11
FN: 16
TP: 94
FP: 83
20 cm/sec
TN: 6
FN: 17
TP: 90
FP: 73
25 cm/sec
TN: 10
FN: 27
TP: 91
FP: 80
Total detection rate
TN: 9
FN: 20
Note: True Positive (TP), True Negative (TN), False Positive (FP), and False Negative (FN)
15 cm/sec
The International Symposium on Agricultural and Biosystem Engineering (ISABE) 2013
Conclusion
Both the color and NIR band images could be obtained from chestnuts with a 2CCD
camera system and the latter were very effective for separation of pin-hole chestnuts which
are evolved and transferred on roller conveyor, indicating detection accuracy of about 85%.
Further study is required to maximize the detection rate of the decayed chestnuts as well as
the pin-hole sample by analyizing the color images.
Acknowledgements
This study was carried out with the support of ´Forest Science & Technology Projects
(Project No. S121010L040120)´ provided by Korea Forest Service.
References
1. Donis-González, I. R., Guyer, D. E., Pease, A., and Fulbright, D. W. (2012). Relation of
computerized tomography Hounsfield unit measurements and internal components of
fresh chestnuts (Castanea spp.). Postharvest Biology and Technology 64(1): 74-82.
2. Fang, J. J., Liu, S. L., and Zhang, H. (2004). Real-time Chestnut Sorting System by
Machine Vision. Light Industry Machinery, 3: 92-93.
3. Food and Agriculture Organization. (2011). FAOSTAT online. Food and Agriculture
Organization of the United Nations. Available at Web site: http://faostat.fao.org/ (verified
July 15, 2013).
4. Hui, Z., Xiaoyu, L., Zhu, Z., Chenglong, W., and Yun, G. (2011). Detection of chestnut
defect based on data fusion of near-infrared spectroscopy and machine vision.
Transactions of the Chinese Society of Agricultural Engineering, 2: 060.
5. Liu, J., Li, X. Y., Li, P. W., Wang, W., Zhang, J., Zhang, R., and Liu, P. (2010).
Nondestructive detection of moldy chestnut based on near infrared spectroscopy. African
journal of Agricultural research, 5(23): 3213-3218.
6. Liu, L., J., Li, X., Li, P., Wang, W., Zhang, J., Zhou, W., and Zhou, Z. (2011). Nondestructive measurement of sugar content in chestnuts using near-Infrared spectroscopy.
In Computer and Computing Technologies in Agriculture IV, 246-254.
7. Wang, C., Li, X., Wang, W., Feng, Y., Zhou, Z., and Zhan, H. (2011). Recognition of
worm-eaten chestnuts based on machine vision. Mathematical and Computer Modelling,
54(3): 888-894.
8. Wu, D., He, Y., Feng, S., & Sun, D. W. (2008). Study on infrared spectroscopy technique
for fast measurement of protein content in milk powder based on LS-SVM. Journal of
Food Engineering, 84(1):124-131.