A hybrid sampling algorithm for highly imbalanced class-overlapping data based on Mahalanobis distance and nearest neighbor
摘要
In many fields, imbalanced data is a common phenomenon that presents challenges for data classification. Current improvement measures for this issue mainly focus on imbalanced classification, overlooking the more serious problem of class overlap. Compared to ordinary imbalanced classification, samples with a high degree of imbalance and overlapping regions often face significant classification challenges and are usually difficult to classify accurately. This paper proposes a hybrid sampling algorithm called MNHS (Mahalanobis distance and Nearest Neighbor-based Hybrid Sampling) for highly imbalanced class-overlapping data to address this issue. The MNHS algorithm first performs undersampling using a comprehensive metric that combines minority class density and K-nearest neighbors. It then identifies, within the neighborhood of a minority class sample, the majority class sample that is closest in density value to serve as an auxiliary instance for constraining the subsequent oversampling process. Finally, MNHS relies on minority class density and K-nearest neighbors to clean the data. The research involved testing various algorithms on a highly imbalanced dataset with significant class overlap to assess their classification effectiveness. The results highlight the superior performance of the MNHS algorithm in terms of Recall, G-mean, and AUC metrics.