Feature selection, also known as variable selection, plays a critical role in statistics, signal processing, and machine learning by enhancing model interpretability, identifying relevant variables for specific tasks, and enabling effective dimensionality reduction. Central to this process is the accurate ranking of variables. In this study, we conduct a systematic and fair comparison of several wrapper-based feature selection methods to evaluate their performance. To ensure a controlled and insightful analysis, we generate synthetic datasets that incorporate variable correlations, different distributions, and interdependencies. Our findings reveal that two specific wrapper methods consistently outperform others, offering practical guidance for both researchers and practitioners. Notably, the widely discussed leave-one-covariate-out (LOCO) method-closely related to Shapley value-based approaches-performs the worst in our evaluations. In contrast, the backward elimination method, which constructs a variable ranking by iteratively removing the least important features, achieves the highest performance. These results underscore the importance of method selection in feature ranking and provide actionable insights for future applications.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Comparison of Ranking Methods for Feature Selection

  • Grace Villacrés,
  • Marius Marinescu,
  • Luca Martino,
  • Óscar Barquero

摘要

Feature selection, also known as variable selection, plays a critical role in statistics, signal processing, and machine learning by enhancing model interpretability, identifying relevant variables for specific tasks, and enabling effective dimensionality reduction. Central to this process is the accurate ranking of variables. In this study, we conduct a systematic and fair comparison of several wrapper-based feature selection methods to evaluate their performance. To ensure a controlled and insightful analysis, we generate synthetic datasets that incorporate variable correlations, different distributions, and interdependencies. Our findings reveal that two specific wrapper methods consistently outperform others, offering practical guidance for both researchers and practitioners. Notably, the widely discussed leave-one-covariate-out (LOCO) method-closely related to Shapley value-based approaches-performs the worst in our evaluations. In contrast, the backward elimination method, which constructs a variable ranking by iteratively removing the least important features, achieves the highest performance. These results underscore the importance of method selection in feature ranking and provide actionable insights for future applications.