Feature selection for support vector machines in imbalanced data

Abstract

Addressing the effects of class imbalance on feature selection models has become an increasingly important focus in academic research. This study introduces a novel support vector machine (SVM)-based algorithm specifically designed to handle class imbalance during the feature selection process. Using the Taiwan bankruptcy dataset as a case study, the algorithm incorporates the ExtraTreeClassifier() to manage class imbalance and identify a reduced set of relevant variables. To validate the selected features, SVM is applied within the imbalanced data context. Subsequently, analysis of variance (ANOVA) ranking is employed to further refine the variable set to three key features. An SVM model tailored for class imbalance is then constructed to assess the effectiveness of the final feature set. The proposed model significantly outperforms existing approaches in terms of classification performance. Specifically, it achieves a Type I error of 1.17% and a Type II error of 22.9%, compared to 4.4% and 39.4% reported in prior research. In terms of overall accuracy, our method reaches 83.1%, surpassing the 81.3% achieved by earlier studies. These results demonstrate that the proposed feature selection algorithm not only improves SVM accuracy but also outperforms other feature selection techniques when used in conjunction with SVMs, particularly under conditions of class imbalance.