2022 International Conference on Electrical Engineering, Computer and Information Technology

University of Jember, Jember, Indonesia, November 22-23 2022

Attractive Learning Media for Introduction to

Popular Fruits Using Computer Vision
Zilvanhisna Emka Fitri Lalitya Nindita Sahenda Rizdky Oktaviari Pratama Putra
2022 International Conference on Electrical Engineering, Computer and Information Technology (ICEECIT) | 978-1-6654-9352-9/22/$31.00 ©2022 IEEE | DOI: 10.1109/ICEECIT55908.2022.10030261

Department of Information Technology Department of Information Technology Department of Information Technology

Politeknik Negeri Jember Politeknik Negeri Jember Politeknik Negeri Jember
Jember, Indonesia Jember, Indonesia Jember, Indonesia
[email protected] [email protected] [email protected]

Zul Arsil Mustofa

Moh. Iskandar Zulkarnain Adinda Ayu Triasasti Department of Information Technology
Department of Information Technology Department of Information Technology Politeknik Negeri Jember
Politeknik Negeri Jember Politeknik Negeri Jember Jember, Indonesia
Jember, Indonesia Jember, Indonesia [email protected]
[email protected] [email protected]

Abstract— Early Childhood Education (ECE) starts from learning media are needed to increase student learning
the age of 1 year to 4 years, this program helps children to motivation [6]. Along with the development of technology,
prepare for school. Because children have a short range of learning media uses multimedia (a combination of several
concentration power so that children's attention will be easily elements such as text, sound, images and animation) [7]
distracted by other things that attract their attention. Learning which helps users to understand the learning material.
media usually use books, cards (flash cards) or items made of Another method is the use of computer vision as an attractive
paper and cause students to get bored quickly, so alternative learning media for early childhood [8].
learning media are needed to increase student learning Fruit recognition is a learning material that is easy to
motivation. Another method is the use of computer vision as an
apply to early childhood. The fruits chosen are fruits that are
attractive learning media for early childhood. Fruit recognition
is a learning material that is easy to apply to early childhood.
quite familiar and have a distinctive shape for children. The
The stages in this research consist of a collection of popular fruit introduction of fruit using two languages or bilingual such as
datasets, converting RGB color space to HSV color space, Indonesian and English. In the introduction of the fruit, the
segmentation process, feature extraction and classification of researchers used a combination of digital image processing
popular fruits using the KNN method. Parameters used for fruit techniques and intelligent systems known as computer vision.
recognition based on color and shape features. The results of the Several reference studies related to the introduction of
system accuracy in recognizing popular fruit types are 100% for fruit as a digital learning media such as fruit recognition
the value of K is 1 for both blue images and saturation images. educational games for kindergarten-age children [9] and the
use of AR as a digital learning media for the introduction of
Keywords—attractive learning media, computer vision, K- fruit [10]–[13]. The similarity of these references is the use
Nearest Neighbor, popular fruit of learning media such as educational games and augmented
reality (AR). Augmented reality (AR) is a technology that is
I. INTRODUCTION able to combine picture cards that will be captured by the
Early Childhood Education (ECE) starts from the age of camera and processed and displayed 3D animation on the
1 year to 4 years, this program helps children to prepare for cellphone screen in real time[10]. But the drawback of this
school [1]. Every child has different abilities and potential AR technology, students can only see 3D objects so visual
[2]. Every child has its own uniqueness in terms of talent, learning styles are applied, while auditory and kinetic
interest, learning style and so on. In addition, children like to learning styles have not been met.
fantasize and imagine [3] so to support early childhood In order to fulfill the three learning styles, the use of
development through the presence of objects and based on the computer vision is made for real-time fruit recognition. So
experience of concrete objects [4]. According to DePorter that the reference that is also used is the introduction of fruit
types in the image using a classification approach based on
and Hernacki (1992) based on the type of information
color and texture features[14]. Besides these two features,
displayed, individual learning styles are divided into three
features such as fruit size and shape can also be used for fruit
categories, namely visual, auditory and kinesthetic. Visual recognition[15]. Then the reference to the use of computer
learning style is a learning style by seeing, observing, vision related to fruit recognition such as the introduction of
looking, and so on. The auditory learning style is a learning fruit types based on color and texture parameters using K-
style by listening while the kinesthetic learning style is a Nearest Neighbor with the highest accuracy of 92% [14] and
learning style using movement, touching or taking action [3]. the Naïve Bayes method is able to classify fruit with an
Methods, media and technology are needed to support the accuracy of 81%[16].
learning process so as to produce a better quality of the Based on the explanation above, the intelligent system
learning process [5]. Because children have a short range of used in this research is the K-Nearest Neighbor (KNN)
concentration power so that children's attention will be easily method. The difference with previous research is the type of
distracted by other things that attract their attention. Learning fruit dataset used. In previous studies, there were 12 types but
media usually use books, cards (flash cards) or items made of there are several types of vegetables such as agata potato,
paper and cause students to get bored quickly, so alternative

1978-1-6654-9352-9/22/$31.00 ©2022 IEEE 14

Authorized licensed use limited to: Universidad Nacional de Colombia (UNAL). Downloaded on August 15,2023 at 20:50:26 UTC from IEEE Xplore. Restrictions apply.
 asterix potato and onion. This study used 10 types of fruit that TABLE I. TYPES AND NUMBER OF POPULAR FRUIT DATASETS
have unique shapes and names that are popular in early No. Fruit Name Amount of data
childhood. Another development is that researchers add 1 Avocado 75
output in the form of text and sound that have been adapted 2 Kiwi 49
to the bilingual language used. 3 Lemon 51
4 Pineapple 61
II. METHODOLOGY 5 Pear 100
6 Banana 28
The stages in this research consist of collection of popular 7 Snakefruit 55
fruit dataset, convert RGB color space to HSV color space, 8 Starfruit 52
color space component separation process, segmentation 9 Strawberry 125
process, feature extraction and classification of popular fruits 10 Dragonfruit 90
using the KNN method as shown in Figure 1. Total amount of data 686

B. Convert to Another Color Space

The original image is an image that has an RGB color
space. Next, the color space is divided into the components
of the color space to find out which color space component
best represents the object of research. In addition, the image
is converted to another color space such as the HSV color

3 ( − )
space [17] using the equation :

=
( − )+( − )
(1)

( , , )
= 1− (2)
+ +
=
3
(3)
After the color space conversion process, the HSV image
is also divided into components such as in the RGB image.
After getting the component image that best represents the
shape of the object, the next process is the segmentation
process. In addition to the segmentation process, the RGB
image is extracted its color features.
C. Segmentation Process
The segmentation process aims to separate the object and
Figure 1. Block diagram of popular fruit recognition system

( , )={ ( , )<" ℎ 1 ( , )
background with the equation (2) :

>" ℎ 0
A. Collection of Popular Fruit Dataset (4)
The data used in this study is primary data taken using
a digital camera and a ministudio box with 3 LED lights as Determination of the threshold value (T) based on the
shown in Figure 2. The image size is 3601x1801 pixels with histogram of the image of the RGB color space component
10 popular fruit types and the amount of data used is and the HSV color space component. In addition to
described in Table I. segmentation based on the gray value threshold, then
segmentation is carried out based on the area threshold value

& = ≥&
(A) [18] with the equation (3) :
'() *+, (5)

D. Feature Extraction
Feature is a distinctive value that can distinguish each
class of popular fruit. In this research, color and shape
features are used. The color features used are red, green and
blue color components while the shape features used are area,
perimeter [18], shape [19] and diameter [20] features using
Figure 2. (a) Types of Fruit and (b) The Process of Collecting Popular the formula equation:
Fruit Datasets

& =. / 0 1ℎ 1 2 (6)

3 =. 4 1 2
(7)
+ 5. 22 1 2 √28

15

Authorized licensed use limited to: Universidad Nacional de Colombia (UNAL). Downloaded on August 15,2023 at 20:50:26 UTC from IEEE Xplore. Restrictions apply.
 3 9
ℎ / =
&
(8) on the histogram image of the blue image and the saturation

; 0 < ℎ+ 0 < ℎ
image as shown in Figure 4.
: =
2
(9) Based on the results of Figure 4, there are two discrete
signals in both the blue histogram and the saturation image.
To get the area and perimeter features using the chain
In the blue image, the gray value is in the range of 120 – 210
code technique where the white object is labeled by with the number of pixels more than 18 x 104, but this value
calculating it based on the 8 neighboring direction [17]. The is not a fruit object but a background. This can be seen from
major axis length value is the furthest distance between the the blue image (Figure 3), while the histogram of the
centroid and the outermost pixel coordinates, while the minor saturation image, the fruit object has a gray value ranging
axis value is the closest distance between the centroid and the from 0.5 to 0.7 with a number of pixels of 0.5 x 105 or 5 x 104
pixel coordinates [20]. based on the saturation image (Figure 3). Then the threshold
value for the blue image is 118 while the threshold value for
E. K-Nearest Neighbor Method
the saturation image is 0.13 so that the segmentation image
The K-nearest neighbor method is a 'lazy' classification results are shown in Figure 5.
method where the data is classified based on the K value
which is calculated based on the closest distance to the
Euclidean distance (ED) calculation using the formula
equation :

'

=:( , ) = >. ( − ;)9 (10)

?@A

Where air is the testing data and aij is the training data.

In general, the KNN algorithm is as follows.

1. Determine the number of neighbors (K) to be used for
class considerations.
2. Calculate the Euclidean distance from the new data to
each data point in the dataset.
3. Take a number of K data with the closest distance, then
determine the class of the new data (a)

III. RESULT AND DISCUSSION

The original image is an image that has an RGB color
space. In addition, RGB images are converted to HSV color
space using the equation (1) - (3). Furthermore, the color
space components are separated by color space components
to find out which color space component best represents the
object of research as shown in Figure 3.

(b)
Figure 4. Histogram (a) Blue Image and (b) Saturation Image

Figure 3. Result of HSV Color Space Conversion and Splitting of RGB and
HSV Color Space Components

Based on Figure 3, in the process of separating the RGB

color space components, the blue component is the best color
space component in representing the fruit object because
there is a significant difference in the gray value between the
object and the background. When the RGB image is
converted to the HSV color space and each color component
is separated, the saturation color component is the color space
component that best represents the fruit object. The next Blue Image Saturation Image
process is the segmentation process using the equation
Figure 5. Image Segmentation Results
formula (4). determination of the threshold value (T) based

16

Authorized licensed use limited to: Universidad Nacional de Colombia (UNAL). Downloaded on August 15,2023 at 20:50:26 UTC from IEEE Xplore. Restrictions apply.
 The next step is the feature extraction process. Feature
The example of a pear, the segmentation results on the blue extraction using color and morphological features using
image are better than the segmentation results on the equation (6) - (9) and the results of these features are shown
saturation image. However, the segmentation results can be in Table II for color features and Table III for morphological
different if other fruits such as kiwi and pineapple are used as features.
shown in Figure 6. TABLE II. AVERAGE COLOR FEATURES ON EACH FRUIT CLASS
Figure 6 (b) shows that both the blue image and the
Class Red Green Blue
saturation image have the same threshold value as Figure 5 Avocado 195 195 190
and produce an imperfect segmentation image that requires Kiwi 176 175 170
an "imfill" code to fill in the blank (hollow) object areas. Lemon 171 170 161
While Figure 6(a) also shows the results of imperfect image Pineapple 179 174 157
Pear 176 173 164
segmentation in both the blue image and the saturation image. Banana 173 171 163
The result of the segmentation image is that there are small Snakefruit 170 170 167
white objects around the fruit object. To overcome this, Starfruit 187 197 191
segmentation is carried out based on the area threshold value Strawberry 166 166 163
Dragonfruit 161 145 146
or the Channel Area Thresholding (CAT) method. Image
segmentation results using a combination of code 'imfill' and TABLE II. AVERAGE MORPHOLOGICAL FEATURES ON EACH
the CAT method produce image segmentation results shown FRUIT CLASS
in Figure 7. Class Area Perimeter Shape Diameter
Avocado 614533 4125,02 32,04 846,57
Kiwi 324171 2250,87 15,06 646,84
Lemon 339533 2569,42 19,51 665,14
Pineapple 2307359 24665,31 273,16 713,66
Pear 684575 3518,89 19,02 919,94
Banana 458288 3926,85 33,83 948,34
Snakefruit 185916 1857,41 18,89 495,97
Starfruit 707939 3312,69 16,30 989,15
Strawberry 83916 1642,56 32,59 351,93
Dragonfruit 1088247 6985,29 46,32 1201,34

(a) From the results of the features in Table II and Table III,
there are 7 features that are used as input from the K-Nearest
Neighbor (KNN) intelligent system. In this study, the amount
of data used is 686 image data consisting of 10 types of
popular fruit which are described in Table 1. Calculation of
the KNN method by calculating the closest distance using the
Euclidean distance calculation in the equation (10), then the
fruit data is grouped (classified) based on the K value. In this
study, we used a value of K = 1 to a value of K = 10 for both
blue images and saturation images, then the system accuracy
results are shown in Table IV.
(b) TABLE IV. THE SYSTEM ACCURACY RESULTS
Figure 6. Image Results of Segmentation (a) Kiwi and (b) Pineapple
Accuracy (%)
K Value
Blue Image Saturation Image
1 100,00 100,00
2 91,84 92,27
3 91,25 90,38
4 87,76 89,80
5 88,19 87,32
6 86,88 87,61
7 86,88 86,30
8 84,84 85,71
(a) 9 85,28 84,26
10 84,40 83,82
11 84,99 84,11
13 85,13 84,69
15 84,55 83,53

Based on the accuracy results shown in Table IV, both

blue images and saturation images with a value of K = 1
produce a system accuracy of 100%, but the accuracy
decreases as the value of K increases. The KNN method is
quite simple in classifying data because the data is sorted
from the shortest to the longest distance is then adjusted based
on the value of K and checked for the dominant or priority
(b) data class, the data sought will be in accordance with the
priority data. If more and more K values are used, the priority
Figure 7. Image segmentation result

17

Authorized licensed use limited to: Universidad Nacional de Colombia (UNAL). Downloaded on August 15,2023 at 20:50:26 UTC from IEEE Xplore. Restrictions apply.
 classes will be many. this causes the possibility of errors in when we use K = 1 then we classify based on one sample
classification. Therefore the feature data from the fruit must which is the nearest neighbor and it can cause overfitting.
be compared with other methods such as Naive Bayes or
Artificial Neural Networks to get the optimal level of The factor that causes it is the possibility that the testing
accuracy. data has a high similarity to the training data and each class
has a clear shape feature. If we use a lot of K values, then this
The display of the introduction of popular fruits using method will classify the class based on the priority class of its
computer vision and we have also implemented the K value, thus allowing for errors in classification. Therefore
application as a Community Service Activity with the the feature data from the fruit must be compared with other
Community Science and Technology Application Scheme at methods such as Naive Bayes or Artificial Neural Networks
KB Nurul Kharomah as shown in Figure 8. to get the optimal level of accuracy.
ACKNOWLEDGMENT
We are grateful to the Politeknik Negeri Jember which
has funded the Community Service Program as a source of
PNBP funds and KB Nurul Kharomah Jember as our partner.
REFERENCES
[1] A. Ansari, R. C. Pianta, J. V. Whittaker, V. E. Vitiello, dan E. A.
Ruzek, “Starting Early: The Benefits of Attending Early Childhood
Education Programs at Age 3,” Am. Educ. Res. J., vol. 56, no. 4, hal.
1495–1523, 2019, doi: 10.3102/0002831218817737.
[2] D. Ayuni dan F. A. Setiawati, “Kebun Buah Learning Media for Early
Childhood Counting Ability,” J. Obs. J. Pendidik. Anak Usia Dini,
vol. 3, no. 1, hal. 1, 2019, doi: 10.31004/obsesi.v3i1.128.
[3] A. Lestariningrum, Perencanaan Pembelajaran Anak Usia Dini.
Nganjuk: Adjie Media Nusantara, 2017.
[4] Guslinda dan R. Kurnia, Media Pembelajaran Anak Usia Dini.
Surabaya: Jakad Media Publishing, 2018.
(a) [5] A. R. Safira, Media Pembelajaran Anak Usia Dini. Gresik: Caremedia
Communication, 2020.
[6] Y. D. Puspitarini dan M. Hanif, “Using Learning Media to Increase
Learning Motivation in Elementary School,” Anatol. J. Educ., vol. 4,
no. 2, hal. 53–60, 2019, doi: 10.29333/aje.2019.426a.
[7] K. D. Ansari dan R. Rosnelly, “Perancangan Aplikasi Game Edukasi
Pengenalan Jenis Buah–Buahan Berbasis Multimedia,” J. Mhs. Fak.
Tek. dan Ilmu Komput., vol. 1, no. 1, hal. 915–926, 2020.
[8] S. Riyadi, Z. E. Fitri, dan A. M. N. Imron, “PENGENALAN HURUF
LATIN PADA ANAK USIA DINI DENGAN PENERAPAN
METODE BACKPROPAGATION,” Djtechno J. Inf. Technol. Res.,
vol. 2, no. 2, hal. 130–137, Des 2021.
[9] D. A. Romadhon dan D. A. Diartono, “Permainan Edukasi
Pengenalan Buah Untuk Anak Usia Tk Menggunakan Construct 2,” J.
Din. Inform., vol. 11, no. 1, hal. 24–30, 2019, doi:
10.35315/informatika.v11i1.8145.
(b) [10] A. Pramono dan M. D. Setiawan, “Pemanfaatan Augmented Reality
Figure 8. (a) Display of Popular Fruit Recognition Applications and Sebagai Media Pembelajaran Pengenalan Buah-Buahan,” INTENSIF
(b) Our Community Service Activities J. Ilm. Penelit. dan Penerapan Teknol. Sist. Inf., vol. 3, no. 1, hal. 54,
2019, doi: 10.29407/intensif.v3i1.12573.
IV. CONCLUSION [11] N. A. R. D. Suwoto, “Aplikasi ‘Pengenalan Buah Dan Binatang’
Berbasis Augmented Reality Sebagai Media Pembelajaran Pada
This study aims to create an attractive learning media for Pendidikan Anak Usia Dini,” J. Warn. Pendidik. dan Pembelajaran
early childhood with materials such as the introduction of Anak Usia Dini, vol. 6, no. 1, hal. 8–14, 2021, doi:
popular fruits using two languages (bilingual), namely 10.24903/jw.v6i1.585.
[12] Y. Maelani et al., “Teknologi Augmented Reality Sebagai Media
Indonesian and English. This application uses computer Pembelajaran Dalam Pengenalan Buah-Buahan (Kasus Paud
vision which is a combination of digital image processing Hidayatul Burhan),” J-SAKTI (Jurnal Sains Komput. dan Inform., vol.
techniques and intelligent systems such as K-Nearest 5, no. 2, hal. 911–924, 2021.
Neighbor (KNN). The steps of the image processing [13] A. Y. Bun, “Pengenalan Buah-Buahan Dalam Bahasa Inggris Melalui
technique consist of HSV color conversion, RGB and HSV Augmented Reality Berbasis Android,” Universitas Putera Batam,
color component splitting process, segmentation process and 2021.
[14] E. K. Ratnasari, “Pengenalan Jenis Buah pada Citra Menggunakan
feature extraction. From the results of the study, it is known Pendekatan Klasifikasi Berdasarkan Fitur Warna Lab dan Tekstur Co-
that the blue image and the saturation image are the best Occurrence,” Inf. J. Ilm. Bid. Teknol. Inf. dan Komun., vol. 1, no. 2,
images that represent the shape of the fruit perfectly with a hal. 88–97, 2016, doi: 10.25139/inform.v1i2.846.
threshold value (T) of 118 for blue images and 0.13 for [15] A. Bhargava dan A. Bansal, “Fruits and vegetables quality evaluation
saturation images. using computer vision: A review,” J. King Saud Univ. - Comput. Inf.
Sci., vol. 33, no. 3, hal. 243–257, 2021, doi:
Advanced segmentation is needed such as Channel Area 10.1016/j.jksuci.2018.06.002.
Thresholding to remove noise in the segmented image. When [16] A. Saputra, “Klasifikasi Pengenalan Buah Menggunakan Algoritma
Naive Baiyes,” J. Resist. (Rekayasa Sist. Komputer), vol. 2, no. 2, hal.
the researcher applies the KNN method, the optimal K value 83–88, 2019, doi: 10.31598/jurnalresistor.v2i2.434.
used is K = 1 to produce a system accuracy of 100%. but [17] Z. E. Fitri, L. N. Sahenda, P. S. D. Puspitasari, P. Destarianto, D. L.
Rukmi, dan A. M. N. Imron, “The Classification of Acute Respiratory

18

Authorized licensed use limited to: Universidad Nacional de Colombia (UNAL). Downloaded on August 15,2023 at 20:50:26 UTC from IEEE Xplore. Restrictions apply.
 Infection (ARI) Bacteria Based on K-Nearest Neighbor,” Lontar [20] Z. E. Fitri, A. Baskara, A. Madjid, dan A. M. N. Imron, “Comparison
Komput., vol. 12, no. 2, hal. 91–101, Agu 2021. of Classification for Grading Red Dragon Fruit (Hylocereus
[18] P. Destarianto, A. N. Noviana, Z. E. Fitri, dan A. M. N. Imron, Costaricensis),” J. Nas. Tek. Elektro, vol. 11, no. 1, hal. 43–49, 2022,
“Detection of Essential Thrombocythemia based on Platelet Count doi: 10.25077/jnte.v11n1.899.2022.
using Channel Area Thresholding,” J. RESTI (Rekayasa Sist. dan
Teknol. Informasi), vol. 6, no. 1, hal. 9–15, Jan 2022, doi:
10.29207/resti.v6i1.3571.
[19] N. E. Paulina, Z. E. Fitri, A. Madjid, dan A. M. Imron, “Klasifikasi
Kerusakan Mutu Tomat Berdasarkan Seleksi Fitur Menggunakan K-
Nearest Neighbor,” MIND (Multimedia Artif. Intell. Netw. Database)
J., vol. 6, no. 2, hal. 144–154, Des 2021, doi:
10.26760/mindjournal.v6i2.144-154.

19

Authorized licensed use limited to: Universidad Nacional de Colombia (UNAL). Downloaded on August 15,2023 at 20:50:26 UTC from IEEE Xplore. Restrictions apply.