Two Phases Feature Selection Method for Classification Problems Based on Association Rules
摘要
Feature selection has become a key issue in data mining because the quality of the selected attributes affects the performance and generalization capabilities of classifiers. The challenge is to reduce the number of features without reducing prediction performance. The ARFS (Association Rules Feature Selection) algorithm will select an ordered vector of features from the association rules and will then use the sequential feature selection (SFS) method on this vector, to select a best subset. However, ARFS cannot consider all features when selecting attributes, it is not applicable on real data and the feature subset selected by ARFS does not depend on the size of the subset. We propose Association Rules Feature Selection larger (ARFSL) which is a two-phase algorithm based on ARFS. In a first phase, SFS selects the attributes according to a confidence threshold of the association rules whose consequent is the class and in which they are found in the antecedent. In the second phase, SFS selects from the remaining features those that optimize the prediction performance of the model when applied to a dataset described only by each of the features. Experimental results show that the proposed algorithm achieves average gains of \(3.02\%\) , \(3.64\%\) and \(5.24\%\) in terms of accuracy, f1, precision respectively, and selects on average fewer attributes compared to state-of-the-art algorithms such as: ReliefF, MI, SFS, RFE-SVM, RIDGE and LASSO.