Betel leaf classification using color-texture features and machine learning approach

ABSTRACT


INTRODUCTION
Betel (Piper betle Linn) is a well-known species in the genus Piper since it is not only utilized as a herb but also has important cultural or cultural value in the community. Betel leaf has traditionally been used as an anti-inflammatory, antiseptic, antibacterial, bleeding stop, cough suppression, laxative fart, stimulant saliva, intestinal worm prevention, itching relief, and sedative [1]. Betel plants are classified according to the color of their leaves; some are green, red, black, and yellow, while others are silver. Betel is classified into four categories based on leaf color: red betel wulug, green betel, golden betel, and black betel.
Red betel (Piper crocatum Ruiz & Pav.) is in high demand due to its medicinal and ornamental characteristics [2]. This plant has a high selling price since its attractive look, particularly its leaves. The red betel plant is a climbing plant that grows on fences and trees. When illuminated, the surface of the red betel leaf is silvery red and reflective. Wulung betel is sometimes referred to as purple betel because it generates a purple glow when lit from below at night. Green betel is typically utilized for traditional rituals and medicinal purposes. Golden betel, also known as betel jalu, features batik-like or pale-yellow patches, whereas black betel is sometimes associated with the supernatural.
Piper betle Linn, also known as tambula (Sanskrit) (Hindi and Bengal) in many countries outside of Indonesia, thrives in the humid tropical climate that dominates Southeast Asia [3]. In traditional medicine, betel leaf aids in the healing of wounds and promotes digestion. Additionally, betel leaf extract possesses antibacterial, antifungal, and anti-inflammatory properties. This plant's components, including piper-betel, piperol A, and piperol B, display platelet-activating factor receptor (PAF) antagonist activity [4]. Currently, numerous approaches to computer vision in agriculture are being developed. Segmentation and classification are the two fundamental processes required to achieve these objectives. In a previous study, thresholding [5], [6], clustering [7], [8], and edge detection [9] were frequently utilized in the segmentation of plants. In addition, several prior studies on agriculture in plants have been conducted. Moreover, in the classification process, K-nearest neighbors (KNN) [10], artificial neural network (ANN) [11], and support vector machine (SVM) [12], [13] were often used methods. Those studies regularly use digital image processing for agricultural classification. Pandurng and Lomte [14] provided four examples of categorization in agriculture: i) crop management, ii) nutrient deficit and plant content identification, iii) crop and land estimation and target tracking, and iv) fruit quality control, sorting, and grading. The study applied image processing and machine learning to categorize fruit and leaf diseases.
Gining et al. [15] presented RGB and HSI color spaces also gray level co-occurrence matrix (GLCM) color as the texture feature for harumanis mango leaf disease detection. The purpose of disease classification was to justify the type of leaf disease. The proposed method detects and diagnoses the condition with 68.89% of precision. Dubey and Jalal [16] proposed recognizing apple fruit illnesses based on images by implementing the K-means clustering technique and categorizing the images into many disease groups using multi-class SVM. The study extracted the features using the global color histogram (GCH) and color coherence vector (CCV). The result revealed that CCV was superior to CGH. The downside of the K-means clustering method was that the initial K value might influence the experiment's outcome.
Tigadi and Sharma [17] applied deep learning for autonomous banana leaf disease detection to determine the final infection percentage. They extracted the color features using the histogram template (HOT) method. Jeyalakshmi and Radha [18] assert that the proposed software was efficient and able to replace the manual process of plant disease identification. The classification of guava leaf diseases was an additional topic of study. It used various techniques to extract texture features, including scale-invariant feature transform (SIFT), space extrema detection, keypoint localization, orientation assignment, and keypoint descriptor. As classifiers, SVM and KNN are utilized. The result indicated that SVM performs marginally better than KNN in the classification task. The limitation was that it demands a large amount of computation due to several distinct strategies being utilized [19].
This study aims to determine the type of sirih leaves using a practical feature extraction method for color features, including RGB, HSV, HSI, LAB color spaces, and texture features. Three main processes were implemented: image processing techniques, feature extraction, and assessing the effectiveness of five classifiers: the ANN, KNN, naive bayes, random forest, and SVM. The color and texture features have been implemented because of their robustness to differentiate the betel species.

METHOD
The proposed method consists of two phases: training and testing. Each phase consisted of several main processes: image acquisition, region of interest (ROI) detection, pre-processing, feature extraction, classification, and evaluation. The input of this method was a betel leaf image. The leaf was differentiated into three classes. Initially, the ROI detection process and pre-processing were carried out to simplify the following processes. Afterward, feature extraction was applied to generate color and texture feature values to identify the characteristics of the betel leaf. In the final stage, classification was performed to determine the betel leaf type based on the image data. The sequence of these processes is depicted in Figure 1. Meanwhile, the details of each main process are described in the following subsection.

Image acquistion
This process was carried out to collect data consisting of betel leaf images divided into three classes: black, red, and green. This study used several types of equipment, such as a studio minibox using 1 LED strip light with a power of 220 V. The leaf was placed in the center of the minibox studio with the white color of the background. The smartphone camera was standing on a tripod to capture the images. The distance between the leaf and the camera was 20 cm. There were 120 images of each class betel leaf; therefore, 360 images were collected to form the dataset. These images were

Region of interest detection
In order to expedite the subsequent process, the original 4,000×1,844 pixels were reduced to 500×500 pixels, as present in Figure 3(a) [20]. In addition, the process outlined was utilized to generate a sub-image known as the ROI image, which focuses on including the leaf area as a region of interest. The image was cropped and converted from RGB to grayscale color spaces. Furthermore, thresholding with the Otsu method [21] was performed to estimate the betel leaf area utilized as the ROI image boundary, as shown in Figure 3(b). The size of the generated ROI image changes due to the leaf's varied widths, as illustrated in Figure 3(c).

Pre-processing
In this study, color and texture were proposed as features. The RGB color spaces were converted into HSV, HSI, and LAB color spaces, as shown in Figures 4(a)-(c). Additionally, The RGB image was transformed to grayscale, as depicted in Figure 4(d). The resulting images of the conversion process were necessary to produce the color features. Meanwhile, grayscale images were required to extract the texture feature. The conversion of RGB to HSV, HSI, and LAB color spaces was performed as in [11].

Features extraction
This procedure generated the feature value used to discriminate between betel leaf classes. The feature values were retrieved from the color and texture. The integration of color characteristics was necessitated by the ability to differentiate between various types of betel leaves. In order to extract color features based on RGB, HSV, HSI, and LAB color spaces, color moments were performed. Meanwhile, texture features were necessary since each betel leaf has a distinct texture. These features were produced with the GLCM method against the grayscale images. The following subsections discuss the detail of feature extraction methods.

Color moments
The color features were successfully extracted using color moments [22]. Most color distribution information was included in the low-order moments. This study utilized five distinct types of color moments to extract the features. The color distribution is represented by the mean (μ), standard deviation (σ), skewness ( 1 ), entropy (S), variance ( 2 ), and kurtosis ( ) values. The first through sixth orders were utilized to express this distribution. These values were determined in (1)- (6). The color features were derived by extracting these features from all channels of the RGB, HSV, HSI, LAB, and grayscale color spaces.
Here, N is the number of pixels in the image, and Pij is the value of pixel j at color component i.

Gray level co-occurrence matrix
GLCM is a way to find the gray level that appears most often in pairs of pixels at a certain distance (d) and angle orientation (θ) by analyzing each pixel in the image. Most of the time, 0°, 45°, 90°, and 135° are used. The (i,j)th entry in the GLCM matrix shows how often the gray level I is followed by the gray level j with a distance of d and an angle of (P). Four features were used in this study: contrast (X1), correlation (X2), energy (X3), and homogeneity (X4). Those texture features were computed based in (7)-(10) [23]:  Table 1.

Classification
The final process of the training and testing phase was classification. In this study, the feature values produced by the subsequence process were used as the input data. Several machine-learning approaches were implemented to classify betel leaf types based on color and texture features. There were four classifiers applied, including KNN, naïve bayes, SVM, and ANN, due to the robustness to classify various objects [24]- [29].

Evaluation
A confusion matrix is a machine learning framework that contains information about a classification system's actual and predicted classifications. A confusion matrix has two dimensions: the actual item class and the class predicted by the classifier. Figure 5 illustrates the fundamental construction of a confusion matrix for multiclass classification problems with classes A1, A2, and An. Nij represents the number of samples belonging to class Ai that was misidentified as belonging to class Aj in the confusion matrix [30]. This study employed three evaluation metrics to assess the performance of the proposed method: precision, recall, and accuracy. Those parameter values were determined using the multiclass confusion matrix. The training and testing data were separated using cross-validation with a k-fold value of 5. The evaluation parameters were computed using (12)-(14) [22]:

RESULTS AND DISCUSSION
The betel leaf classification method was developed to determine three kinds of betel leaf: black, green, and red, where each class consist of 120 images; therefore, the total number of data was 360 images. This method was based on color and texture features by applying four machine learning approaches KNN, ANN, SVM, and Naïve Bayes. The method's performance was evaluated based on three parameters: precision, recall, and accuracy. Cross-validation with k-fold value 5 was performed to separate the training and testing data. The evaluation result of the developed method based on color features, texture features, and color-texture features was summarized in Tables 2-4, respectively.  Tables 2 and 3 show the classification results based on color and texture features, respectively. Those features were fed into different classifiers. Both features achieved similar classification results with accuracy values of 98.9% for ANN, Naïve Bayes, and KNN, while the highest accuracy value was obtained using SVM of 99.44%. Meanwhile, the classification results based on the combination of color and texture features are shown in Table 4. The Naive Bayes classifier obtained an accuracy value of 98.9%, while the ANN and KNN produced 99.4%. The SVM classifier successfully achieved the maximum accuracy value of 100%. It indicates the combination feature of color and texture appropriate ad powerful for the dataset used in this study.  The detail of the classification result is depicted in Figure 6. It shows misclassification occurs in the black and red betel class. Both were classified as green betel leaves, with the number of misclassification data being one image, as shown in Figures 6(a) and (b). The color feature causes the black betel leaf to be classified as green betel leaf. In contrast, the texture features induced the red betel to be classified as green betel leaf. Furthermore, Figure 6(c) presents all class that was successfully classified using color and texture features.
This study proposed a method to classify the betel leaf into three classes. The evaluation result achieved precision, recall, and accuracy values of 100%. It indicates no misclassification occurred using the combination features of color and texture integrated with the SVM classifier. The suitability in selecting the features and classifier has an important role due to the color and texture features without the SVM classifier being unable to obtain the maximum accuracy value of 100% and vice versa. Furthermore, the number of data has an essential role because more training data make the method learn more patterns; therefore, the classification results will increase.

CONCLUSION
This study developed the method of classifying betel leaves into three classes: green betel, black betel, and red betel. The method consists of five main processes: image acquisition, ROI detection, pre-processing, feature extraction, and classification. Image acquisition was acquired 360 betel leaf images utilized for training and testing. The Otsu thresholding method was applied in the ROI detection, followed by feature extraction. The features were extracted based on color and texture features using color moments and the GLCM method. The four machine learning approaches were applied in the classification, including KNN, naïve bayes, ANN, and SVM, using cross-validation with k-fold values of 5. The method achieved a maximum accuracy value of 100% by combining the color and texture features also the SVM method. Based on the evaluation results, the proposed method was suitable for the dataset used in this study.