See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/327887505

Computer Assisted Counter System for Larvae and Juvenile Fish in Malaysian
Fishing Hatcheries by Machine Learning Approach

Article in Journal of Computers · January 2016

DOI: 10.17706/jcp.11.5.423-431

CITATIONS READS

17 925

4 authors, including:

Valliappan Raman Sundresan Perumal

Coimbatore Institute of Technology Universiti Sains Islam Malaysia (USIM)
66 PUBLICATIONS 505 CITATIONS 72 PUBLICATIONS 440 CITATIONS

SEE PROFILE SEE PROFILE

Siti Fazilah
Universiti Tunku Abdul Rahman
17 PUBLICATIONS 27 CITATIONS

SEE PROFILE

All content following this page was uploaded by Sundresan Perumal on 13 January 2020.

The user has requested enhancement of the downloaded file.

 Journal of Computers

Computer Assisted Counter System for Larvae and

Juvenile Fish in Malaysian Fishing Hatcheries by Machine
Learning Approach

Valliappan Raman1*, Sundresan Perumal2, Sujata Navaratnam3, Siti Fazilah3

1 Multimedia Research Group, School of Computer Science, Universiti Sains Malaysia, Malaysia.
2 Faculty of Science and Technology, Universiti Sains Islamic Malaysia, Malaysia.
3 Department of Computing, KDU University College, Malaysia.

* Corresponding author. Tel.: 0060164404636; email: [email protected]

Manuscript submitted April 4, 2015; accepted November 20, 2015.
doi: 10.17706/jcp.11.5.423-431

Abstract: The increased in number and size of larvae and juvenile growth are estimated based on manual
approach in fishing hatcheries. There is a high demand for computer assisted software solution for
aquaculture research in early detection and recognition of fish population. There exist several companies
who have introduced fish detection technologies into the market. Although able to count the number of
larvae with a high accuracy rate, the fish counter software’s may encounter difficulties when detecting
smaller larvae’s and ants in very early stage of birth period. The main aim of the paper is to propose a
framework using machine learning techniques that can be of low cost and efficient system for fish counting
and growth study. The expected final result will be a complete preliminary prototype with basic camera
setup, focus on larval fish. medium term, improve camera setup and quality; focus on larval and juvenile fish.
For the Long term, full fish growth tracking and data mining is implemented. The proposed research in this
paper will assist the Malaysian fisheries department to have accuracy on detecting the larvae, juvenile and
ants in the hatcheries.

Key words: Aquatic, enhancement, segmentation and counting.

1. Introduction
The early life history of fish is often considered critical for recruitment to the adult population. Thus,
understanding how larval fish population’s distribution varies through time and among habitats is
important for answering questions related to both applied and theoretical ecology in marine research. The
global demand for aquaculture software, supplies and equipment is predicted to exceed $80 billion by 2016
due to increasing demand for fish in the market. Commercialization of farming across the world and
increasing intensities of production will necessitate the use of commercial grade feeds, chemicals and
biological enhancements as well as more mechanization on fish farms. Therefore, demand for fish is
expected to continually outpace growth in other global food sources such as livestock and agriculture. The
Asia Pacific region is projected to account for over 80 percent of global demand for aquaculture software
supplies and equipment by 2016. This is in direct correlation to the region’s global dominance in
aquaculture production. Several factors contribute to the huge aquaculture production turnover from Asia:
a long coastline; tropical and temperate weather suitable for rearing aquatic species; low labor costs in a
hitherto labor intensive industry; strong local demand as well as export competitiveness; integrated

423 Volume 11, Number 5, September 2016

 Journal of Computers

farming practices enabling multiple crops, better land and resource utilization; and a growing
manufacturing base for supplies and inputs. Due to the increasing exploration on the fishery industry, new
method had to be explored in order to support it and to keep it sustainable. Machine learning approach
might give solution for supporting the fishery industry. For instance, it can help to detect, recognize, and
analyze the fish by using image processing techniques and it will be faster, easier and convenient than the
manual way. But there are several problems in detecting and recognizing the fish in the image such as the
target object is obscured due to the presence of the other object which can interfere with recognition
process and any other noise. It makes the system difficult to identify the feature of the fish stages.
In fishing hatcheries, when the reproduction system of fish breeding starts from egg to spawning stage.
Each fish species has a unique reproductive strategy and favors certain habitats for spawning and for early
development of their newly hatched young. Fish life cycles vary among species. In general, however, fish
progress through the following life cycle stages: 1) Eggs: Fertilized eggs develop into fish. Most eggs do not
survive to maturity even under the best conditions. Threats to eggs include changes in water temperature
and oxygen levels, flooding or sedimentation, predators and disease. 2) larval: Larval fish live off a yolk sac
attached to their bodies. When the yolk sac is fully absorbed, the young fish are called fry. 3) Fry: Fry are
ready to start eating on their own. Fry undergo several more developmental stages, which vary by species,
as they mature into adults. 4) Juvenile: The time fish spend developing from fry into reproductively mature
adults varies among species. Most fish do not survive to become adults. Threats to survival include
fluctuations in water temperature, changes in oxygen levels, competition for habitat and predators. 5) Adult:
When fish are able to reproduce, they are considered adults. 6) Spawning: Female fish release eggs into the
water and male fish fertilize eggs by releasing milt. Not all eggs are fertilized. Some fish spawn each year
after reaching maturity; others spawn at intervals while others spawn only once and then die.
Our research involves in counting the larvae and juvenile in second and fourth stage of the cycle which is
important for aquatic researchers to monitor the growth of fish by categorized into alive, partially alive, and
not alive. However, the counting process is time consuming and can be subjected to human error. In this
paper, we explore how this counting can be automated using image processing approaches. Early studies
have shown that some possible ways of fish counting. However, they are designed for bodies of water with
different varieties of fish so they include some form of fish species detection algorithm so as to be able to
count the number for each species. In addition, fish tracking algorithms are also used in the counting of fish.
Thus, their suggested methods are computationally intensive and will increase costs and slow down the
counting speed.
The key idea behind our proposed method is the use of camera to mark the position of the fish breeding
in the hatcheries, and then analyzing the area of each image to decide how many larvae’s and juvenile are
captured in each image. This in turns yields the number of fish in a multiple images. Individual images from
various positions are processed separately, and the number of fish computed from each image is then added
up and divided by the total number of images from multiple locations to obtain an average count.
The paper is organized as follows: Section 2 explain the research challenges for counting the early stages
of larvae in the hatcheries. Section 3 describes the research objective for the research carried out in the
paper. Existing works and commercial software products for counting the fish in market is explained in
Section 4. Machine learning approaches are explained in detail at Section 5. The initial results of the
proposed method are shown in Section 6. Finally conclusion is made in Section 7.

2. Research Challenges
Detection and recognizing an object is very difficult and takes a long times especially for the live object. In
fisheries industries, species of the fish must be recognizing for calculating their population. Currently,

424 Volume 11, Number 5, September 2016

 Journal of Computers

manual approach is done by an official of fisheries department one by one for calculating and measuring
fish length. One of the fundamental challenging problems in computer vision is detecting object inside an
image or video frames. Therefore monitoring larval fish production in freshwater assesses the overall
effectiveness of habitat restoration in lakes and rivers across the nation have challenges as follows: 1)
Aquaculture research, to calculate the livestock of larvae, manually performed by human. 2) Count accuracy
rate was very poorly determined. 3) High error margin with less performance result.4) Larva growth
information’s are not properly maintained and integrated.

3. Research Objectives
According to the Department of Fisheries Malaysia statistics, the market value of marine fishes have
increased significantly from the year 2000 (RM 4 billion) to 2009 (RM 6 billion) with a growth rate of
around 50% [1]. Furthermore, based on the Tenth Malaysia Plan, high value agriculture activities including
aquaculture shall be focused upon [2].From this plan, the government plans to introduce modern
technology to improve production processes as well as intensify research and development, in hopes to
create new high-value added products. These points give reason to the creation of an aquaculture software
solution. The research aims to count and measure the total number of larvae fish through image processing
techniques. This provides users with accurate data concerning larvae production which in turn leads to
better planning and decision making. Additionally, the software based larvae detection system provides
improved reliability, efficiency and accuracy when estimating the total number of fish. The main research
objectives leads for the development of larvae counter system are as follows:
 Develop a machine learning algorithm to Count and Measure the number of larvae fish and to
calculate livestock value, food requirements and cost.
 Provide the user accurate information about the production or catch process and so facilitate
planning and decision-making.
 Achieve higher accuracy rate with good performance and reliability.
 Deliver larvae information report on reproduction and adult stages.
The proposed research solution, the fish counter intends to address several existing issues regarding
larval and juvenile fish counting.

4. Existing Works
There are few existing research works carried out in the past for counting the fishes based on the legth
and size of measurement in adult stage not in young stage. The research carried out in the past are by using
image processing method. In 2006, D. J. White [3] developed a machine system for sorting some fish species
by employing modern hardware and programming techniques in real time. The system is called as Catch
Meter, which includes conveyor, light box and feeder. The mechanical systems were controlled by a PLC
interfaced to the main computer and software via an Ethernet link. Fish moved along the conveyor belt at
1.5 m/s and was analyzed by the computer vision system. Fish detection also was done using camera video
like Williams’ experiment [4]. Researcher constructed a prototype system for detecting and delineating fish
shapes in video footage obtained from a local aquaculture facility. Another method is to use edge detection
to identify the fish [5]. The canny edge detector is used because it provides the most number of true edges
[6]. To eliminate the edges of the container, the image is filled with redundant black data starting from the
sides of the image outwards. This set up an additional rectangular edge denoting the original outer border
of the image. Morphological closing is done next to join the edges that were not terminated properly and to
fill in the areas. Blobs are thus created from the edges. This step is also necessary to connect the round
edges of the container to the original outer border of the image so they can all be eliminated by eliminating

425 Volume 11, Number 5, September 2016

 Journal of Computers

blobs which has a major axis much longer than the length of five fish. The final results are obtained after
eliminating noise and background objects. This results in inaccuracies when we use the area to count the
number of fish in a blob. These methods are not promising to produce accuracy and reliability in counting
the number of fishes from hatcheries.
There are few commercial products available in the market to count the fish from the different type of
environment. Several competing companies have introduced fish counting and monitoring solutions that
enable the detection of fishes through specialized equipment. These tools range from light counters which
detect fish through underwater lights, to sound waves which count the number of fish. Although capable of
accurately counting fish, these options fail to detect fish under certain measurements (e.g. below 5cm).
Furthermore, the cost of acquiring such equipment may be high (e.g. around $75,000 and above). Vaki, an
Icelandic company, allows for fish counting by a light valve. When a fish passes their counter, light is blocked,
therefore indicating that a fish is present. Although able to count fish with a high accuracy rate, the Vaki fish
counter may encounter difficulties when detecting smaller fish. Furthermore, it may function incorrectly
when used in turbid water [7]. The company, Sound Metrics, provides Dual frequency identification sonar
(DIDSON) which detects fish through sound waves. Underwater pictures are produced which are later
processed with fish recognition software. Although high quality images can be produced, the DIDSON
module also faces issues when counting small (below 5 cm) fish. Additionally, the cost for each DIDSON unit
is expensive with prices from $75,000 and above [8]. AKVA Group with their Fishtalk software enables
users to store aquaculture data as well as create reports. Moreover, the software is not species dependent
meaning that it can be compatible with numerous fishes. Although this is so, the Fishtalk software does not
specifically support monitoring larval fish, and therefore may omit collecting certain important information
[9]. The current work differs from these existing products in terms of cost as well as hardware and software
usage. Though other solutions offer specialized tools, they must be directly purchased and may incur a high
price. Additionally, most of these tools are not able to accurately estimate the larval fish population due to
their reduced size. In contrast, the proposed prototype can function with any normal camera that provides
sufficient picture quality, consequently reducing equipment costs. Furthermore, custom larvae detection
software enables the counting of small larval fish which may remain unrecognized by other solutions [10]
[11].
Furthermore, research has been undertaken to improve support towards fisheries which leads to better
aquaculture development. The main benefits from the outcome of this research include fish farmers,
hatcheries or marine researchers who require low cost solutions in improving their fish production or
aquaculture research. Since the outcome of this research is a prototype which only requires a camera and
recognition software by using machine learning approaches, it is fit for any research experts or industry
wanting to detect and study larval and juvenile fishes.

5. Methodology
Larvae and Juvenile detection and recognition system is an image processing technology used to detect
the young larves and juvenile in an image and recognize the number of larves and juvenile present in an
image. Therefore the main aim of this paper is to segment the images into separate single image and it is
counted. Segmentation is used for detecting the edges in an image. The purpose of segmenting an image
into a set of disjoint areas/segments is to simplify the image. The image may be further simplified by
assembled the small areas into larger objects by building a symbolic scale space over the segments. The
segmentation in this image is entirely based on the gradient transformation. Segmentation and the choice of
segmentation algorithm cannot be proven optimal to the task of recognizing objects in any strict sense. The
gradient transform has proven itself useful in many contexts, but the choice is heuristic and can only be

426 Volume 11, Number 5, September 2016

 Journal of Computers

justified by experience. In the case of recognition of objects such as young larvae and juvenile in images,
the object must in some sense be different from its surroundings. Therefore the methodology is categorized
in four stages: Image Acquisition, Image Enhancement, Segmentation and Classification, in final
classification are performed with counting the number of young larvae and juvenile fishes in the captured
image. Below Fig. 1 illustrates the working methodology of machine learning approach.

Fig. 1. Illustrates the methodology of machine Fig. 2. Illustrates the image acquisition of capturing
learning approach. the larvae image from hatcheries.

5.1. Image Acquisition

Images were captured via camera mounted on the aquatic basin (See Fig. 2). Once the camera captures
the larvae, it will be transferred to the video software for processing the image. The captured image is
converted into grayscale image by color transformation method.

5.2. Image Enhancement by Sobel Operator

Larvae and Juvenile can be easily detected in an image if the larvae and Juvenile have sufficient contrast
from the background. In order to improve contrast of larvae edges and separate from ants, Sobel operator is
applied in the square of the image, which creates a binary mask using a user-specified threshold value.
Multiple young larvae are present in the square of image, so that the larvae can be seen in its entirety. The
object to be segmented differs greatly in contrast from the background image. Changes in contrast can be
detected by operators that calculate the gradient of an image.

5.3. Segmentation by ROI Hunting and Edge Detection

First step in the segmentation is to identify the region of interest. Each image containing a square field
was processed in order to establish a region of interest. The captured image is processed using a simple
algorithm that is able to detect the four vertices of the square field. Horizontal coordinates of these vertices
are estimated by observing local minima on a profile function which is obtained with the sum of pixel values
over each column of the image. Vertical coordinates are obtained similarly but with a profile function
considering the sum over each row of the image.
Second step is to pre-segment the edges in the image. The binary gradient mask shows lines of high
contrast in the image. These lines do not quite delineate the outline of the object of interest. Compared to
the original image, gaps can be viewed in the lines surrounding the object in the gradient mask. These linear
gaps will disappear if the Sobel image is dilated using linear structuring elements. Therefore the gaps are
filled. Nest stage where the binary gradient masks is dilated using the vertical structuring element followed
by the horizontal structuring element. The dilated gradient mask shows the outline of the larvae cell and
ant quite nicely, but there are still holes in the interior of the cell, therefore the gaps are filled in uniform

427 Volume 11, Number 5, September 2016

 Journal of Computers

format.
The larvae and juvenile cell of interest and ant has been successfully segmented, but it is not the only
object that has been found. Any objects that are connected to the border of the image need to be removed.
The connectivity was set to 4 to remove diagonal connections, so that objects are separated into individuals
and it is detected in order to make the segmented object look natural; smooth the object by eroding the
image twice with a diamond structuring element, so that each and every larvae cells and ants are detected
and it is easy to count the number of larvae’s and ant in image. Therefore while overlapping or
contradiction appears in an image, it is detected by boundary detection algorithm. Therefore if there are
multiple larvae are overlapped, then considered two nodes and by that the two nodes are replaced by one,
their mutual edge is ignored, and the neighbors of the new node is the union of the two original nodes
individual neighboring sets. Instead of contracting the two nodes imposed by a connected operation, a new
node can be introduced, and the two nodes can both be linked to the new node by edges. Then the full set of
nodes (both original and new ones) and the set of edges alone constitute a tree. So that overlapped nodes
are separated and it counted easily into one full object in an image.

5.4. Classification and Counting Method

Larvae and Juvenile in the image were classified depending on whether they had contact with each other.
It is identify, whether the larvae were entirely inside the region of interest or not from the image. Two
processes were undergone in the stage; those larvae that were not connected in any way with the border
edges and those that were connected in some way. The process of classification is based on neural network
approach. The method is divided into two main components: firstly, the recognizing system that uses neural
network, and secondly the scanning method that examines each location in the image at several scales,
looking for locations that might contain a larvae and juvenile fish. The neural network that is used in this
system is multilayer perceptron. The architecture used is based on the work that done by previous work [8].
Multilayer perceptron neural network is one of the conventional feed-forward artificial neural networks.
The multilayer perceptron used has 400 input units, which receive input from 20 × 20 grayscale pixel from
window scanner. The feature used for this work is the edges of the larvae. This is used for the first
experiment purpose for this work. When the larvae and juvenile are detected, usually the object would be
detected for several times. This makes the detection result are overlapped each other. The object detected
should be in one detection area. Therefore the detected objects should be merged. The object candidates are
merge by the considering the number of overlapping detection in the method.
Counting of Larvae- Each object is categorized as a single larva or juvenile or a multiple larvae that are
connected with each other in the image. Pixel counting was used to calculate each object’s area. System
processes an image in rows and columns, a pixel is at the intersection of a line and column and the system
stores a value for each pixel. Therefore 0 is for no larvae present (black) and 255 for larvae present (white).
While determining the area of a larva, then it is needed to calculate the number of larvae/juvenile contained
in all images. It is assumed that the area of different objects contained in images can either be of a single
larva or of a multiple number of the area of a single larva. While calculating the larvae, histogram is
calculated. The histogram shows a periodic behavior that is caused by the fact that multiples of the area of a
single larva will be the main objects present in all images. The period in this histogram is considered as a
valid interpretation of the area of one larva. Second stage is to apply sobel operator to enhance the contrast
from background to detect the larvae. ROI is calculated and boundary detection algorithm is applied to
segment and count the number of larvae's/juvenile through connected regions.

6. Initial Experimental Results

A testing conducted to give a view of its performance in detecting young larvae and juvenile fish [12]. A

428 Volume 11, Number 5, September 2016

 Journal of Computers

preliminary experiment was carried out to determine the accuracy rate of the method. Images were
captured via camera mounted on an aquatic basin. Once the camera captures the larvae image, it will be
transferred to the counter system for image processing. The container acts as the background while the fish
are the foreground objects to be extracted. Therefore the container must be bright in color to provide an
acceptable contrast to the fish. A video camera records images of the fish for a predefined amount of time.
The captured image is then converted into a grayscale image by color transformation methods,
enhancement method, segmentation and classification. The data used in this testing divided into two data
which are the training data set for training, and the testing data set for the testing. The neural network
architecture used in this work is multi-layer perceptron. The system will be given 400 input unit of 20 × 20
pixel grayscale image of the sub-image. The output value from the unit will be around -1 and 1. In the
training data set, the value 1 is defined as the larvae edges, while -1 is defined as the non larvae. The result
shows that the counter gives a number of false positive and false negative. Some area was detected and
somewhere not detected, which need to further developed for accuracy. Currently accuracy achieved in
detecting larvae is 82% and juvenile is 87%.
A simple setup for practical implementation is performed as explained above, Larvae and Juvenile
required to be counted flow from a smaller container. These images of larvae and juvenile are processed to
produce the count of the fish. The first set of experiment is implemented in MATLAB. The results of the
machine learning approach in detecting and counting the larvae and juvenile is shown below from Fig. 3 to
Fig. 6.

6.1. Original Image and GUI Interface

Fig. 3. Illustrates the original image and GUI of the prototype.

6.2. Image Enhancement

Fig. 4. Illustrates the noise elimination and contrast enhancement by preprocessing stage.

429 Volume 11, Number 5, September 2016

 Journal of Computers

6.3. Segmentation

Fig. 5. Illustrates the segmentation by edge detection method.

6.4. Classification and Counting

Fig. 6. Illustrates the classification by neural network approach and counting method.

7. Conclusion
This research presents one of the object detection techniques for detecting young larvae and juvenile
fishes from hatcheries. This work is aimed to study the implementation of machine learning technique in
detecting the larvae/juvenile object in a digital image. The result shows that the system is affected by the
training data and the fish image that obscured and has a lot of noise. In future work, certain measures can
be taken to improve the accuracy of classification and counting the missed larvae’s and juvenile fish.

Acknowledgment
We would like to thank the research centre, KDU university college, Malaysia for funding the project
under internal grant scheme for developing the prototype. Would like thank for the facilities and support
provided by faculty of computing, KDU for the project.

References
[1] Official Portal: Department of Fisheries Malaysia. Department of Fisheries Malaysia. From
http://www.dof.gov.my/59
[2] Belongie, S., Malik, J., & Puzicha, J. (2002). Shape matching and object recognition using shape contexts.
IEEE Trans. Pattern Anal. Mach. Intell., 24(4).
[3] White, D. J., Svellingen, C., & Strachan, N. J. C. (2006). Automated measurement of species and length of

430 Volume 11, Number 5, September 2016

 Journal of Computers

fish by computer vision. Fisheries Research, 80, 203-210.

[4] Williams, R. N., Lambert, T. J., Kelsall A. F., & Pauly, T. (2006). Detecting marine animals in underwater
video: Let’s start with salmon. Proceedings of the Twelfth Americas Conference on Information Systems,
Acapulco, Mexico.
[5] Boles, W. W., Geva, S., & Busch, A. (1999). An image processing approach for estimating the number of
live prawn larvae in water. Signal Processing and Its Applications, Brisbane, Australia.
[6] Lee, D. J., Schoenberger, R., Shiozawa, D., Xu, X. Q., & Zhan, P. C. (2004). Contour matching for a fish
recognition and migration monitoring system. Proceedings of SPIE Optics East, Two and
Three-Dimensional Vision Systems for Inspection, Control, and Metrology II.
[7] Broersen, J. (2009). Towards a Detection and Recognition System for Freshwater Fish (MSc Report).
[8] Pontuala, H. de, Roberta, R., & Minerb, B. P. (1998). Study of bivalve larval growth using image
processing. Journal of Aquaculture Engineering, 17(2).
[9] Johnson, I., Harman, M., Forrow, D., & Norris, M. (2001). An assessment of the feasibility of using image
analysis in the oyster embryo-larval development test. PubMed, Environ Toxicol, 16(1), 68-77.
[10] Borenstein, E., & Ullman, S. Class-specific, top-down segmentation. ECCV, 2.
[11] Felzenszwalb, P. F., & Huttenlocher, D. P. (2005). Pictorial structures for object recognition. International
Journal of Computer Vision, 61(1).
[12] Valliappan, R., Brian, L., & Patrick, T. (2011). First prototype of aquatic ToolKit: Towards low-cost
intelligent larval fish counting in hatcheries. Proceedings of 9th International Conference on Pervasive
Intelligence and Computing (pp. 193-195), Sydney, Australia.

Valliappan Raman is a research fellow at Multimedia Research Group Lab, Universiti Sains
Malaysia, Penang, Malaysia. He have worked as a team member to acquire many external
research grants and published papers in impact factor journals. His research interests are
in medical imaging, data mining and health informatics.

Sundresan Perumal is a senior lecturer in Universiti Sains Islamic Malaysia. He acquired

various external research grants and published papers in impact factor journals. His
research interests are in cyber security, medical imaging, data mining and network forensics.

Sujata Navaratnam is a senior lecturer at KDU University College, Malaysia. Her research
interests are in image processing, security, ICT in education and software engineering.

Siti Fazilah Shamsudin is senior lecturer at KDU University College, Malaysia. Her
research interests are in software engineering, human computer interaction and computer
science education.

431 Volume 11, Number 5, September 2016

View publication stats