The development and usability test of an automated fish counting system based on CNN and contrast limited histogram equalization

,


INTRODUCTION
Analyzing information about the underwater environment is important for preventing marine threats, preserving the ocean ecosystem and resources [1].In recent years, the aquaculture industry has expanded due to the adoption of marine ranching techniques.Various work that utilizes artificial intelligence in aquaculture can be found in the literature, which includes fish classification [2], water quality monitoring [3], fish behaviour detection [4], and feeding control [5].Accurate counting of fish populations in fish farms is critical Bulletin of Electr Eng & Inf ISSN: 2302-9285  The development and usability test of an automated fish counting system based on … (Leong Jing Mei) 1129 for proper scientific feeding, sale, export, and control of breeding density [6], [7].Whenever the number of fish decreases, it could indicate non-optimal conditions occurred at the farm and requires the farmers to check all the relevant parameters (e.g. the food source, temperature, and oxygen).A quick response to any issues pertaining to the decrease in the number of fish may ensure that production will not drop significantly.The handling of fish seed in fish farms is labour-intensive, and prone to disruption by many factors such as the weather and water clarity [8].It may take eight workers to count approximately 5 million fish seeds in one day [9].Challenges faced include variations of sizes of the fish and also the complexity of the background environment [10].For example, in University Malaysia Sabah (UMS) fish hatchery, manual fish counting is still taking place.As a result, other activities such as fish feeding, and cleaning of the fish tanks may be delayed.A fish farm that utilizes an automated system to count fish from an underwater image may improve the effectiveness of its management [11], [12], which subsequently increases production.
Underwater images often suffer from distortion and light attenuation due to the scattering of light as it passes through the water [13], [14].Thus, image enhancement is important for accurately detecting the presence of fish in a single image.Once the visual presentation of the image is enhanced, the detection of fish is performed.Various techniques were used; the most common are blob detection and deep learning based.A number of work for automated fish identification and counting is presented in [15]- [17], but they are either tested in a research setting or developed as a standalone application, which limits access to the system.A deep learning architecture, you only look once (YOLO), has shown efficiency in object detection [18].A fish counting system should deploy an effective technique for image enhancement and efficient fish detection to increase accuracy.
This paper presents a deployed intelligent fish counting system that uses underwater images is presented.The contributions of this paper are: i) contrast limited adaptive histogram equalization (CLAHE) was used to improve the presentation of the images and YOLOv5 to identify and count the fish in the images and ii) a web-based prototype was developed to test the intelligent fish counting system, whereby the usability evaluation of the prototype was conducted on hatchery workers at University Malaysia Sabah fish hatchery.
This paper is organised as follows.Some background to the topics related to fish counting and the method proposed are described in section 2. In section 3, the results and analysis are presented.The conclusion and suggestion for future work are outlined in section 4.

MATERIALS AND METHOD
In this section, brief reviews of the materials used are described.The methodology used is also presented.

Contrast limited histogram equalization
Histogram equalization (HE) is a process that creates a new version of an image with a more balanced distribution of intensity values.This is achieved by rearranging the intensity values of the original image in order to expand the range of values in the image histogram.The CLAHE method is a variant of HE.It limits the range of intensity of the output image to prevent outliers in the histogram that avoids oversaturation of regions with high intensity values.
To perform CLAHE, first, the histogram of the intensity values in the image is calculated.This is followed by the transformation of the image using a cumulative distribution function.CLAHE is a commonly used method for enhancing images [19], and more information about it can be found in [20].

YOLOv5
Unlike other complex object detection networks, YOLOv5 uses a single neural network to the image, then separates it into regions and predicts bounding boxes and confidence values for each region.Weight and biases are directly integrated into YOLOv5, providing experiment metric tracking, model and dataset versioning, and rich model prediction visualization.The class probabilities are used to weigh these bounding boxes.It requires only one forward propagation pass through the neural network to make predictions.It generates detected target objects together with the bounding boxes after non-max suppression.YOLOv5 can predict several bounding boxes as well as their class probabilities.When training on images, the YOLOv5 network learns generalizable representations of objects, outperforming existing detection approaches.
YOLOv5 uses cross stage partial networks (CSPNet) as its backbone to extract informative features from an input image.It also minimizes model parameters and floating-point operations per second (FLOPS), resulting in improved model speed and accuracy while reducing the model size [21].A path aggregation network (PANet) is applied as the model neck to reduce processing time.PANet employs a novel feature pyramid network (FPN) topology with an improved bottom-up approach to enhance the propagation of lowlevel features [21].Feature pyramids aid models in generalizing when it comes to object scaling.Figure 1 shows the YOLOv5 architecture employed in this paper [21].A cross stage partial network (CSP) is the basis of the architecture.It has several convolutional layers (conv) and concatenate functions (concat).The pooling

Convolutional neural network for fish counting
A deep convolutional neural network (CNN) is used to extract and learn complex yet abstract features of the input video for object detection [22].For object detection, object segmentation is needed to locate the target in the image extracts from the underwater video.The architecture is built to process images with 224x224 pixels, adequate for extracting useful features within reasonable computational time.It can capture fish of varying sizes that appear simultaneously in images.The method will select the region of interest (ROI) in the input image's feature space to detect the target objects in the image based on their texture, size, and shape features.
An automated fish counting approach based on a dilated convolution neural network (DCNN) model is presented to accomplish the real-time counting of the salmon population [17].In the proposed model, convolution kernels of various sizes adapt to the fish movement changes in angle, shape, and length.It can minimize the loss of spatial structure information during network transmission.Regression methods are then used to learn the relationships between image features to solve counting in large-scale noisy scenes and generate a density map.
Transfer learning is used to address the limited dataset issue in [23] for fish species classification.Google inception-v3 pre-trained model was selected.There are three different approaches that were examined in the work, including a pre-trained model approach that used Inception-v3 as the feature extractor and the support vector machine (SVM) as the classifier.This approach produced the best results.
A blob-based analysis for fish counting can be found in [12], [24], [25].The user is required to input an image from a computer folder into the system [12].Using a MATLAB graphical user interface (GUI), the edge detection and blob analysis were conducted to identify the number of fish in the image.The fish counting model produced good accuracy when applied to a small, medium and large samples dataset.Based on the implementation presented, the system is developed as a standalone application, thus limiting the mobility of the system.In the actual settings of a fish hatchery, application mobility would be an added advantage.
Although a number of similar works can be found in the literature, all the presented work focused more on the research point of view, whereby the main aim was to produce the fish counting model with the best accuracy.In the context of the proposed work presented in this paper, we evaluate the fish counting performance and also the usability of the system at the real fish hatchery to ensure the model is actually usable by the targeted users.To the best of our knowledge, none of the previous work has tried to measure how well the developed model could be used in a real fish hatchery setting.

Method
The methodology consists of four phases; i) requirement analysis, ii) development of the fish counting engine, iii) development of the prototype of the web-based system, and iv) usability testing (UT).In the first phase, a literature review on the existing fish counting system, and work is conducted to identify the necessary features for a fish counting system.An interview with the target user, in the context of this paper is the UMS IPMB's fish hatchery, is conducted and analyzed.The output of this phase is the required specifications of an automated fish counting system.ISSN: 2302-9285  The development and usability test of an automated fish counting system based on … (Leong Jing Mei) In the second phase, the intelligent fish counting engine is designed and developed.Figure 2 shows the framework of the model generation of the fish counting engine.First, the image is captured and enhanced using CLAHE.Next, the enhanced image is fed to the YOLOv5 architecture to generate the fish identification and counting model.The model generation is repeated several times using different epochs to identify the best model.The output of this phase is the fish counting model.

Figure 2. The flowchart of the fish counting model generation
In the third phase, the design and development of the web-based automatic fish counting system were performed.Figure 3 shows the use case diagram of the proposed system.The user can update and even delete their profiles if they want to.The automatic fish counting function requires an image to be inserted into the system, either using the user's phone camera or selecting an image from the existing repository.Once the image is inserted, the fish counting is performed, and the result is displayed to the user and stored in the database.The user can view their past fish counting activity through the view past result module.The prototype of the proposed intelligent system is the output of this phase.
In the fourth phase, UT was conducted to get the users feedback on the automatic fish counting system.Questionnaires were distributed to the target users to get their feedback on the usability of the developed automatic fish counting system.

RESULT AND DISCUSSION
In this section, the outcomes of the requirement gathering, the implementation of the proposed fish counting system, and the usability testing are presented.

Requirement analysis
In this section, the system requirement is presented.To gather the necessary requirement, a review of the existing systems and an interview with the targeted user have been conducted.From the activities, the following features should be included in the proposed system: -The system must be accessible on a desktop as well as on a mobile phone.In some cases, the fish hatchery worker may need to count the number of fish in the field, for example an ad hoc counting conducted at the fish tank; hence the system may be accessed using a mobile phone.-A module to enhance the presentation of the image.Due to illumination variations caused by the lighting conditions during the image acquisition, a mechanism to enhance the image presentation should be made available.-A module whereby the user can access their past fish counting for record purposes.

Fish counting model analysis
In this section, the performance of the automatic fish counting model is analyzed.First, the analysis of the image enhancement conducted using CLAHE is presented.Second, the accuracy of the automatic fish counting model is described.The deepfish [26] dataset is used to generate and test the model.

Image enhancement analysis
To measure the effectiveness of CLAHE in enhancing the underwater image used in work described in this paper, a visual and statistical inspection is performed.Figure 4(a) is the original image and Figure 4(b) is an example of the enhanced image.As shown in Figure 4, CLAHE enhances the brightness of the foreground object without darkening the background in all cases, making the foreground more distinct.CLAHE makes all the fish objects more visible than the original image.This is important as the YOLOv5 architecture, which is used to detect and count the fish, is based on object detection which requires higher edge visibility of the object of interest.To evaluate the effectiveness of image enhancement techniques, three common statistical metrics to assess the quality of an image were utlised: entropy, normalized root mean squared error (NRMSE), and absolute mean brightness error (AMBE).The optimal entropy range for improving image quality is between 7.0 and 8.0 [27].A lower NRMSE indicates a better contrast, which is desirable as it allows for better discrimination of the foreground from the background.An enhanced image with a low AMBE value shows less variations of its brightness compared to the original image.

Bulletin of Electr Eng & Inf
ISSN: 2302-9285  The development and usability test of an automated fish counting system based on … (Leong Jing Mei) 1133 Table 1 show the examples of the enhanced images produced performances that are close to the value of the optimal metrics as described in the foregoing paragraph.The statistical analysis is aligned with the analysis of visual inspection shown in Figure 4. Hence, the CLAHE has improved the underwater images, which is required before fish detection and counting can be performed effectively.

Fish detection and counting model analysis
To generate the automatic fish detection and counting model, two public datasets were used, the FISH20 project [11] and the automatic zebrafish tracking project [28].A total of 2132 images were used, whereby 80% were for training and 20% for testing.F1-score is used to measure the performance.YOLOv5 was deployed with Adam optimizer.Figure 5 shows the training loss of the model, the x-axis represents the number of epochs and the y-axis the loss value.  2 shows the fish counting F1-score when applied to the test data, where the value was increased until when the epoch was set at 70.A higher F1-score indicates that the system can detect as many fish as possible from the image.The results show that the proposed work is able to produce a good fish counting application, although more work is required to increase the performance.

Prototype demonstration
In this section, the implementation of the prototype is presented, whereby some of the GUI of the system is described.Figure 6 presents the homepage of the proposed FiCo fish counting system.On this page, users are allowed to take a photo or upload an image to perform the fish counting.Once the image is uploaded, the fish counting module will commence after the user clicks the 'calculate now' button.After the fish counting is completed and the result is generated, the user will be redirected to the result page as shown in Figure 7(a).The result page shows the uploaded image, the resulting image whereby the fish in the image is identified and segmented, and the fish count.The generated result will be automatically saved to the database labelled with the user's ID.The user can retrieve the past fish counting activities through the history of the fish counting page shown in Figure 7(b).The user can retrieve their fish counting result on this page whenever necessary.

Usability testing
In the UT, participants were selected and given access to the prototype.Their feedback on the usability of the system was collected using an evaluation form.We tested the usability of the prototype at the UMS fish hatchery, and identified two subjects who were tasked to perform manual fish counting at the hatchery.Figure 8 shows the UT rating of the prototype using the Likert scale of 1-5, whereby 5 being strongly agree.
From Figure 8, the navigational, user-friendliness and instructions scored 3.5, which show that the system is easy to be used and navigated by a user that does not require comprehensive user training.The functionality score of 3 points indicates the system is providing sufficient basic features to meet the needs of Bulletin of Electr Eng & Inf ISSN: 2302-9285  The development and usability test of an automated fish counting system based on … (Leong Jing Mei) 1135 hatchery workers who are required to do fish counting.The design of the interface, however, was too simplistic and less appealing, with some fonts being inconsistently sized.This is reflected in the score obtained at 2.5.Based on the overall score of 3.2, the proposed system is acceptable but there is still room for improvement in order to increase its usefulness.
Figure 8.The UT results

Limitation and future work
The presented fish counting system has several limitations.First, the dataset that was used to train the model for fish identification and counting was limited.Consequently, the model may not generalize well enough to provide a better estimation of fish detection in an image.Second, the system does not provide a feature to get the user's feedback on the accuracy of the fish counting.Such a feature could be used as a gauge for how well they find the fish counting service provided.Some future work is identified.First, adding more image data with variations of data sources could prove useful in producing a more accurate model.Second, a mobile app version of the fish counting system would significantly improve its usability.The app would make it possible for the user to use the system anywhere in the hatchery and easily access their mobile device's camera in order to take an image.This would also allow them to do so without interrupting their workflow and/or requiring them to leave equipment or tools behind.

CONCLUSION
In this paper, an intelligent fish counting system, FiCo, is presented.The images uploaded to the system are pre-processed using CLAHE.To identify and count the fish in the images, a deep learning architecture in the form of YOLOv5 is employed.The usability of the proposed FiCo is evaluated through its implementation at the fish hatchery in UMS.The results of the usability test indicate that FiCo is a viable solution that can aid hatchery workers in the labor-intensive task of fish counting, which is crucial for effective fish hatchery management.


ISSN: 2302-9285 Bulletin of Electr Eng & Inf, Vol. 13, No. 2, April 2024: 1126-1137 1130 is conducted using the spatial pyramid pooling (SPP).The input images are supplied through the backbone.The PANet was used to perform feature fusion.Finally, the detection was conducted at the YOLO layer.

Figure 3 .
Figure 3. Use case diagram of the fish counting system

Figure 4 .
Figure 4. Visual comparison of the image enhancement result: (a) original image and (b) the corresponding enhanced image using CLAHE

Figure 5 .
Figure 5. Training loss of the model

Figure 6 .
Figure 6.The homepage of the FiCO fish counting system

Figure 7 .
Figure 7.The FiCo system pages for; (a) fish counting result page and (b) the history of the fish counting page

Table 1 .
CLAHE performance for image enhancement

Table 2 .
Fish counting performance