Smart Harvesting System with Autonomous Classification of Date Fruit Type and
Maturity Stages Using Deep Transfer Learning
Zainab Abuowda, Shorouk Ramadan, Nour Salam, Jawad Yousaf,
Abdalla Gad, Mohammed Ghazal
Electrical, Computer, and Biomedical Engineering Department, Abu Dhabi University, Abu Dhabi, UAE
Abstract
Proposed System
Results
Date Type Classification:
• Data set contained 11300 images (30% for validation, 15 % for testing)
for each type
• Network settings: number of iterations:13800, epochs: 200
• Test Accuracy: 98% (Resnet-50), 97.3% (AlexNet) and 98.9% (VGG16).
• We proposed an effective deep transfer learningbased system for automatically classifying:
➢ 5 distinct date types on bunches (Naboot saif,
Khalas, Barhi, Meneifi, and Sullaj).
➢ 5 different maturity stages (Immature, Khalal,
Khalal w Rutab, Pre Tamar, and Tamar).
• The study’s findings will be used to design an
intelligent robotic harvesting mechanism for
efficiently monitoring and harvesting date fruits in
a natural environment.
• We used a transfer learning approach on pre-trained
deep convolutions neural networks.
• AlexNet, Resenet-50, and VGG-16 networks were
trained on the available dataset for the two
classification tasks.
Barhi
Kahals Meneifi Naboot Sullaj Precision
Saif
Barhi
1122
4
3
5
2
98.7%
Kahals
0
730
0
0
0
100%
Meneifi
0
2
513
1
0
99.4%
Naboot
Saif
Sullaj
4
0
2
712
0
98.89%
3
0
24
2
1733
98.35%
94.65% 98.89% 99.7%
98.89%
Problem Statement
Recall
99.38% 99.19%
Table 1: Confusion Matrix for Date Type Classification using VGG-16 Network.
• Limited number of high-quality cultivars.
• Lack of proper research on pests and diseases.
Maturity Stage Classification:
• Data set contained 11300 images (30% for validation, 15 % for testing)
for each type
• Network settings: number of iterations:13800, epochs: 200
• Test Accuracy: 96.61% (Resnet-50), 97.87% (AlexNet) and 98.17%
(VGG-16).
• Inadequate harvesting as per modern technologies.
• Lack of experienced and educated employees.
Dataset Examples
Immature Khalal Khalal w’
Rutab
Actual pictures
of Barhi date
Actual pictures
of Sullaj date
Pre
Tamar
Tamar
Precision
Immature
1531
3
3
1
2
99.42%
Khalal
0
476
7
0
0
98.55%
Khalal w’
Rutab
1
45
2043
15
1
97.1%
Pre Tamar
0
0
2
419
4
98.59%
Tamar
Recall
0
99.94%
0
90.84
%
0
99.42%
1
96.1%
87
92.5%
98.86%
98.17%
Table 2: Confusion Matrix for Maturity Stage Classification using VGG-16 Network.
References
• H. Altaheri, M. Alsulaiman, M. Faisal, and G. Muhammed, “Date fruit dataset for automated
harvesting and visual yield estimation,” 2019. Available: https://dx.doi.org/10.21227/x46j-sk98
• S. Wang, X. Xia, L. Ye, and B. Yang, “Automatic detection and classification of steel surface
defect using deep convolutional neural networks. metals, 11 (3), 1–23,” 2021.
Conclusion
• A vision system is proposed for the classification of 5 date types during
their 5 maturity stages using deep transfer learning techniques.
• Future work:
➢ Improving the ResNet-50 model testing accuracy.
➢ Expanding the data set to include videos along with images