See discussions, stats, and author profiles for this publication at: [Link]

net/publication/379986263

Recognition and separation of fresh and rotten fruits using YOLO algorithm

Conference Paper · April 2024

CITATIONS READS

0 159

5 authors, including:

Prem Bahadur Rana Chet Narayan

Khwopa Engineering College Visvesvaraya National Institute of Technology
11 PUBLICATIONS 46 CITATIONS 1 PUBLICATION 0 CITATIONS

SEE PROFILE SEE PROFILE

Ganesh Ram Dhonju

Khwopa Engineering College
2 PUBLICATIONS 11 CITATIONS

SEE PROFILE

All content following this page was uploaded by Ganesh Ram Dhonju on 21 April 2024.

The user has requested enhancement of the downloaded file.

 International Conference on Technologies for Computer, Electrical, Electronics & Communication (ICT-CEEL 2023)

Recognition and Separation of Fresh and Rotten

Fruits Using Yolo Algorithm
Prem Bahadur Rana* Chet Narayan Mandal* Ujjwal Dahal*
Department of Electronics and Department of Electronics and Department of Electronics and
Communication Communication Communication
Khwopa Engineering College Khwopa Engineering College Khwopa Engineering College
Bhaktapur, Nepal Bhaktapur, Nepal Bhaktapur, Nepal
prem2056rana@[Link] mandalchetnarayan@[Link] ujjwaldahal57@[Link]

Kushal Shrestha* Ganesh Ram Dhonju

Department of Electronics and Department of Electronics and
Communication Communication
Khwopa Engineering College Khwopa Engineering College
Bhaktapur, Nepal Bhaktapur, Nepal
kushals2075@[Link] [Link]@[Link]
Abstract— Fruit quality evaluation is crucial in today's It becomes difficult to manage agriculture in a
food processing and distribution systems to assure consumer sustainable way as the human population grows. By 2050,
safety and minimize food waste. The fruits are often sorted the world's population is projected to increase by 40% to
manually using visual examination, which is time-consuming, 9.7 billion people, necessitating a doubling of fruit output.
labor-intensive, and prone to error. To address these issues,
we can use the capabilities of computer vision and Deep
It is anticipated that the number of people employed in
learning to build a powerful and real-time fruit quality agriculture would decline by half by 2050, leading to a
evaluation system. This paper presents an innovative shortage of 5 million harvesters. Hence, more than 10% of
approach for the automated detection and classification of fruits in the world cannot be picked; this is equivalent to the
fresh and rotten fruits through object detection. For our annual consumption of the European Union [3]. The
project, we collected and annotated the images of two fruits: classification of fresh and rotten fruits is generally done by
"Apple" and "Tomato" in different orientations and shades. human, which is inefficient for fruit growers. We can used
The dataset is divided in four classes and is used to train robots for this purpose as they do not get tired from
YOLO algorithm model. This paper presents a novel performing the same task repeatedly like humans do.
approach for the automated recognition and separation of
fresh and rotten fruits on conveyor belts using the YOLO Fruit growers are facing a growing labor shortage as the
algorithm and Raspberry Pi. The integration of computer workforce's interest in agriculture has declined. The
vision and Deep learning techniques empowers the system to problem is worsened by international travel restrictions
perform real- time fruit quality assessment, offering
recently imposed during the pandemic, which have sharply
significant benefits in terms of accuracy, speed, and waste
reduction. This research bridges the gap between technology
limited agricultural productivity due to a lack of migrant
and agriculture, showcasing the potential of AI-powered workers. As a result, tons of fresh produce went
solutions to revolutionize food quality inspection processes. unharvested and rotted in the Felds, where farms had long
depended on foreign seasonal workers [4].
Keywords— Fruits Detection, Dataset, You Only Look
Once (YOLO), Deep Learning, Conveyor belt, Computer Robotic harvesting can offer a possible solution to this
Vision. problem by reducing labor costs (longer durability and
good repeatability) and improving fruit quality assessment.
*Author with *are equal contributors For these reasons, interest in the use of agricultural robots
I. INTRODUCTION in fruit and vegetable harvesting has grown over the last
three decades [5].
Fruits and vegetables are essential components of our
everyday diet. There are many varieties of eatable fruits and The use of intelligent harvesting robots to replace or
vegetables in the natural world. Fresh fruits are not only assist manual harvesting of fresh fruit is significant in terms
wonderful to eat, but it also contains a lot of vital vitamins of reducing production costs and improving economic
and minerals. In the food processing industry, fresh fruits returns. As a typical representative of an agricultural robot,
are used to make delicious and healthy food products [1]. the fruit harvesting robot is considered to have good
potential for the future of intelligent agriculture and has
Automatic fruit classification is an intriguing issue in received widespread attention worldwide [6].
the fruit growing and retailing industry chain because it can
assist supermarkets and fruit growers in identifying various Identification of defective fruit and classification of
fruits and their status from the stock or containers in order fresh and rotten fruit is one of the most important
to increase production efficiency and, ultimately, business challenges in agriculture. Rotten fruits can damage other
profit. As a result, in the past ten years, research into fresh fruits if they are not properly graded and rotten fruits
intelligent systems combining computer vision and are made by hand, which is an inefficient and lengthy
machine learning techniques for fruit defect recognition, process for farmers [7].
ripeness grading, and categorization has been conducted
[2].

247
 International Conference on Technologies for Computer, Electrical, Electronics & Communication (ICT-CEEL 2023)

Therefore, it is necessary to develop a new classification annotation was done by creating boundary boxes around the
model that reduces human effort, cost, and production time images of the fruits. The fruits were labeled accordingly to
in the agricultural industry by identifying fruit defect [8]. their visible characteristics whether they were "fresh/rotten"
or "tomato/apple". Robo flow was used to create bounding
The continued development of machine learning (ML) box and label the dataset with classes "Fresh apple, Rotten
has led to significant advances in agricultural tasks. Deep Apple, Fresh Tomato and Rotten Tomato".
learning (DL) is widely used in fruit recognition and
automatic harvesting because it can extract high-
dimensional features from fruit images. In particular,
convolutional neural networks (CNN) have been shown to
achieve accuracy and speed comparable to humans in some
Raspberry pi
areas of fruit recognition and automatic harvesting.
Compared with digital image processing and traditional
ML techniques, fruit detection and recognition methods
based on CNN have great advantages in terms of accuracy Web Conveyor
[9]. Camera belt
In recent years, the technology of deep learning in
image classification, object detection and face Yolov7-tiny
identification and many other computers vision tasks have model
Fruits
achieved great success. Experimental data shows that the
Collector
technology of deep learning is an effective tool to pass the
man-made feature relying on the drive of experience to the
learning relying on the drive of data [10].
Fig. 2. Detection of Fresh or Rotten Fruit in Conveyor Belt
Through our project, we have focused on automation in Using Raspberry Pi
recognition and separation out fresh and rotten fruits using
YOLO algorithm. YOLO is a fast, accurate object detector, We started by labeling on pictures of fruits to teach our
making it ideal for computer vision applications [11]. computer what they look like. Then, we made a special
learning tool using a powerful computer chip called a GPU
We connect YOLO to a webcam and verify that it from our college. This chip helped our computer get really
maintains real-time performance, including the time to good at recognizing different fruits.
fetch images from the camera and display the detections.
By using dual web camera to capture maximum view of the Next, we took our smart program and put it into a tiny
fruits along with conveyer belt to sort out rotten and fresh computer called a Raspberry Pi. This little computer is
really handy and can do lots of different things. It became
fruits.
the brain of our fruit sorting system.
II. METHODOLOGY To talk to the Raspberry Pi, we used a special program
called PuTTY. It let us give instructions to our system using
words on a computer screen. This made it easier to set things
up just the way we wanted.
Collection Yolov
Labeled Training
of fruits 7-tiny We needed to see the fruits in real-time, so we used web
data
images data
model cameras. They're like the eyes of our system, taking pictures
of the fruits as they move along. These cameras are really
good at capturing clear images, which helps our system do
Fig. 1. Block diagram of training model its job well.
For our project, we first collected the images of fruits: To move the fruits along, we made a moving platform
"Tomato and Apple". The images were classified in four called a conveyor belt. It helps them travel smoothly
groups; "Fresh apple, Rotten Apple, Fresh Tomato and through our system. The conveyor belt was powered by a
Rotten Tomato. A total of 8000 images of apples and special motor called a "Stepper Motor". It's good at moving
tomatoes were collected. We used 2000 images for each things just a little bit at a time, which was perfect for our
class. We collected most of the dataset from sites like conveyor belt.
'Kaggle' and 'Rob flow'. Some of the images were taken by Then the important part - separating the fresh fruits
ourselves. ". We annotated the images in 'Rob flow', and from the not-so-fresh ones. We used a special motor called
divided the dataset in two: "Train and Test". We trained our a "Servo Motor" for this job. This motor gently sorted each
dataset on YOLOv7-tiny. fruit into the right place, making sure only the really fresh
Robo flow is a Computer Vision developer framework
for better data collection to pre-processing and model
training techniques. It also provides datasets for public use.
Kaggle is also another platform where we can find
necessary datasets.
After the collection of data, a text document of annotated
images was generated for creating a training model. The

248
 International Conference on Technologies for Computer, Electrical, Electronics & Communication (ICT-CEEL 2023)

Start 0.1
0.09

Input from camera 0.08

0.07

Train Loss
Pre-processing 0.06
0.05

Fruits Detection 0.04

0.03
0.02
0.01
Are the
fruits 0
Fresh? 0 20 40 60 80 100
Epochs

Sent Fruits to Sent fruits to Fig. 4. Loss Graph of YOLOv7-tiny Mode

Fresh fruits Rotten fruits
collector helped us create an accurate and efficient object detection
collector
model.
Figure 4 shows the graph of loss representation of our
project while training the data. A loss graph is a visual
representation that shows how the error between a model's
predictions and actual data changes as the model learns from
training data. The graph plots the loss value (a measure of
prediction error) on the y-axis and training iterations or
End epochs on the x-axis. As training progresses, the loss value
ideally decreases, indicating that the model is improving its
Fig. 3. Flow chart of detection of rotten or fresh fruits in predictions.
conveyor belt by Using Raspberry Pi
ones ended up in their own special container.
The flowchart shows the steps taken for recognizing and
separating rotten and fresh fruits; the two fruits being Apple
and Tomato. First, the image is taken through web-cameras
when the fruit is being transported by conveyor belt. This
captured image is then given as input to the processor. If and
when, the fruit is detected, the algorithm classifies among
which of the classes, it belongs to. If the fruit is rotten then,
it is sent to the rotten fruit collector. If the fruit is fresh, then
else it is sent to the fresh fruit collector. Separation of fruit
is done using servo motor after the fruit is detected by
camera and is classified
III. EXPERIMENTATION
We trained our custom dataset using a pre-trained model
in YOLOv7-tiny. Images size were kept at 640x640, and
batch size was kept to 64. We fine-tuned the learning
process with specific settings: momentum at 0.937, weight
decay at 0.0005, and an initial learning rate of 0.001. This Fig. 5. F1_Curve
training was done on a powerful GeForce RTX 3080 Ti The F1 curve analysis demonstrates that the model is
GPU with 16GB of RAM. achieving a high level of performance, with an F1 score of
The progress of the model's learning was tracked with a 0.96 at the chosen threshold of 0.526.
loss graph, shown in Fig. 4. This graph gives us a visual These results indicate the model's capability to effectively
overview of how well the model was adapting to the data classify instances across all classes while maintaining a
throughout the training process. Overall, this approach strong balance between precision and recall. The high F1
score suggests that our model is effectively capturing true

249
 International Conference on Technologies for Computer, Electrical, Electronics & Communication (ICT-CEEL 2023)

positive instances while minimizing false positives across threshold is set at 0.5. This suggests that our model is
all classes at the specified threshold. highly effective at recognizing objects even when they
partially match the ground truth.

The [email protected]:0.95 value of 0.826 illustrates that our

model maintains strong performance across a broader range
of overlap thresholds, from 0.5 to 0.95. While the
performance slightly decreases compared to the [email protected]
result, it still demonstrates that our model is adept at
identifying objects with varying degrees of precision and
recall, considering stricter matching criteria.

The high F1 score, impressive AUC-PR value, and

outstanding mAP value collectively indicate our project's
proficiency in recognizing fresh and rotten fruits. This
technological solution has the potential to revolutionize the
fruit quality assessment process, benefitting consumers and
Fig. 6. PR Curve the food industry alike.
Figure 6 shows the graph of the PR curve. The PR curve
(Precision-Recall curve) is a graphical representation that We employed NEMA 23 stepper motors as part of our
shows the trade-off between precision and recall for automated fruit quality detection system. These motors
different probability thresholds in a binary classification or were selected for their capability to provide precise and
multiclass classification problem. It's often used to evaluate controlled movement to our conveyor belt setup. The
models, especially when dealing with imbalanced datasets. conveyor belt's continuous rotation was achieved through
the use of a stepper motor, with the A4988 driver
The resulting PR curve can be used to evaluate the controlling its operation. We introduced a time delay of
performance of the model, and help determine the best 0.003 seconds for the conveyor belt's movement.
threshold for objects detection in the specific application.
An [email protected] value typically ranges between 0 and 1,
We collected some fresh and rotten fruits (i.e., Fresh
where higher values indicate better performance.
Apple, Fresh Tomato, Rotten Apple, Rotten Tomato).
An [email protected] of 0.986 suggests that the model's object These samples were then placed on the conveyor belt.
detection predictions are highly accurate, with a strong Then, the web camera captured images of the fruits as they
balance between precision and recall at the IoU threshold of moved through the system. We then processed these
0.5. These results reflect the model's ability to make images using the YOLOv7- tiny image processing
accurate predictions while maintaining a good trade-off algorithm, which had been pre-trained to identify fruit
between different evaluation metrics. conditions. Our experiments involved quantifying the
system's accuracy in correctly classifying the fruits,
including the percentage of correctly identified fresh and
1 rotten apples and tomatoes.
0.9
0.8 Then, web camera captured real-time images of the
0.7 fruits as they moved along the conveyor belt. These images
0.6 were then analyzed by the system, utilizing algorithms to
mAP

0.5 determine the quality of the fruits whether they were fresh
or rotten. Subsequently, the system sent commands to the
0.4
servo motor to physically separate the fruits accordingly.
0.3
0.2 IV. RESULT AND DISCUSSION
0.1 The evaluation of our object detection model using mAP
0 metrics demonstrates its high accuracy in identifying
0 50 100 objects. The achieved mAP values indicate its proficiency
in discerning objects, even in cases of overlap. This robust
Epoch performance positions our model as a reliable choice for
tasks requiring precise object detection, offering practical
[email protected] [email protected]:.95 applications across a range of real-world scenarios.
This means it could be really useful in places like
grocery stores or factories where quick and accurate sorting
Fig. 7. mAP of YOLOV7-tiny Model is important.
The [email protected] value of 0.986 indicates that our model
performs exceptionally well in recognising objects
accurately, achieving a high level of agreement between
predicted and actual object locations when the overlap

250
 International Conference on Technologies for Computer, Electrical, Electronics & Communication (ICT-CEEL 2023)

The real time recognition and separation of “Fresh

Apple”, “Fresh Tomato”, “Rotten Apple” and “Rotten Web camera Stepper
Tomato” results are as shown in Figure 8, Figure 9 motor
showcasing the practical effectiveness of our model.

Conveyor
belt

Fig. 8. Real time detection of Rotten Apple and Rotten Servo

Tomato motor used
as sorting

Fig. 10. Complete Design of Hardware

Fig. 9. Real time detection of Fresh Apple and Fresh

Tomato V. CONCLUSION
The above Fig. 10. shows the complete design of We specifically utilized Apple and Tomato fruits. These
hardware for recognition and separation of rotten and fresh two types of fruits with four classes “Fresh Apple”, “Fresh
fruits; the two fruits being Apple and Tomato. Tomato”, “Rotten Apple”, “Rotten Tomato” were
subjected to our automated system for the purpose of
In our project, it's important to acknowledge that the use
detecting and categorizing their quality on the conveyor
of a low-cost web camera and the relatively slow processing
belt. Using YOLOv7-tiny image processing algorithm, we
speed of our system did present certain limitations.
successfully developed a system capable of analyzing fresh
The low- cost web camera might have affected the and rotten fruit images for accurate sorting. Integration of
image quality and resolution, potentially impacting the servo and stepper motors played a crucial role, enabling the
system's ability to precisely analyze and distinguish separation of undesirable fruits and continuous belt
between fresh and rotten fruits. This could potentially affect movement. The A4988 driver facilitated precise motor
how accurately our system can analyze and tell the control. This project showcases the potential of combining
difference between fresh and rotten fruits. computer vision, robotics, and Raspberry Pi for enhancing
sorting efficiency and product quality in various industries
A higher- resolution camera with better image quality
would likely improve the accuracy of fruit classification,
enabling the system to make more reliable judgments.
Likewise, upgrading the processing hardware to a faster ACKNOWLEDGMENT
platform would enhance the system's responsiveness and We would like to express our sincere gratitude and
real-time operation. respect to the Department of Electronics and
Taking steps to address these limitations by considering Communication Engineering for the cooperation and
equipment upgrades could be a valuable next move in opportunity to do this meaningful project. We want to thank
refining and optimizing our system for even better Khwopa Engineering College and Khwopa College of
performance. This way, we can ensure that our fruit sorting Engineering for their outstanding help in providing us with
process becomes even more effective and reliable in a GPU for our research. Their support has really made our
practical applications. work better and more successful.
REFERENCES
[1] S. Jana, R. Parekh, and B. Sarkar, “Detection of
Rotten Fruits and Vegetables Using Deep

251
 International Conference on Technologies for Computer, Electrical, Electronics & Communication (ICT-CEEL 2023)

Learning,” 2021, pp. 31–49. doi: 10.1007/978-

981-33-6424-0_3.
[2] T. B. Shahi, C. Sitaula, A. Neupane, and W. Guo,
“Fruit classification using attention-based
MobileNetV2 for industrial applications,” PLoS
One, vol. 17, no. 2 February, Feb. 2022, doi:
10.1371/[Link].0264586.
[3] E. Vrochidou, V. N. Tsakalidou, I. Kalathas, T.
Gkrimpizis, T. Pachidis, and V. G. Kaburlasos,
“An Overview of End Effectors in Agricultural
Robotic Harvesting Systems,” Agriculture
(Switzerland), vol. 12, no. 8. MDPI, Aug. 01,
2022. doi:10.3390/agriculture12081240.
[4] H. Zhou, X. Wang, W. Au, H. Kang, and C. Chen,
“Intelligent robots for fruit harvesting: recent
developments and future challenges,” Precision
Agriculture, vol. 23, no. 5. Springer, pp. 1856–
1907,Oct. 01, 2022. doi: 10.1007/s11119-022-
09913-3.
[5] I. Sa, Z. Ge, F. Dayoub, B. Upcroft, T. Perez, and
C. McCool, “Deepfruits: A fruit detection system
using deep neural networks,” Sensors
(Switzerland), vol. 16, no. 8, Aug. 2016, doi:
10.3390/s16081222.
[6] Y. Li, Q. Feng, T. Li, F. Xie, C. Liu, and Z. Xiong,
“Advance of Target Visual Information
Acquisition Technology for Fresh Fruit Robotic
Harvesting: A Review,” Agronomy, vol. 12, no. 6.
MDPI, Jun. 01,2022. doi:
10.3390/agronomy12061336.
[7] S. S. S. Palakodati, V. R. R. Chirra, Y. Dasari, and
S. Bulla, “Fresh and rotten fruits classification
using CNN and transfer learning,” Revue
d’Intelligence Artificielle, vol. 34, no. 5, pp. 617–
622, Oct. 2020, doi: 10.18280/ria.340512.
[8] A. Bhargava and A. Bansal, “Fruits and vegetables
quality evaluation using computer vision: A
review,” Journal of King Saud University -
Computer and Information Sciences, vol. 33, no. 3.
King Saud bin Abdulaziz University, pp. 243–257,
Mar. 01, 2021. doi: 10.1016/[Link].2018.06.002.
[9] F. Xiao, H. Wang, Y. Xu, and R. Zhang, “Fruit
Detection and Recognition Based on Deep
Learning for Automatic Harvesting: An Overview
and Review,” Agronomy, vol. 13, no. 6, p. 1625,
Jun.2023, doi: 10.3390/agronomy13061625.
[10] G. Zhu et al., 16th IEEE/ACIS International
Conference on Computer and Information Science
(ICIS 2017) : proceedings : May 24-26, 2017,
Wuhan, China.
[11] J. Redmon, S. Divvala, R. Girshick, and A.
Farhadi, “You Only Look Once: Unified, Real-
Time Object Detection.” [Online].Available:
[Link]

252

View publication stats