This work explores the use of the Genetic Algorithm with Linkage Learning (GAwLL) for the feature selection problem. A notable byproduct of GAwLL application for feature selection is the generation of a variable interaction graph, offering insights into feature dependencies. To extend the original C++ implementation—which was limited to the K-nearest neighbors (KNN) algorithm—we introduce PyGAwLLfs, a Python-based framework designed to support feature selection for a wide range of machine learning models. Using PyGAwLLfs, we assess GAwLL performance across various models, including decision trees, random forests, and artificial neural networks. Furthermore, based on the internal mechanisms of GAwLL, we present a novel method for estimating variable importance in machine learning. Experimental evaluation on multiple datasets, including one from particle physics, demonstrates PyGAwLLfs effectiveness in the estimation of variable importance and interaction. We compare the variable importance scores generated by PyGAwLLfs with those produced by traditional approaches.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Estimating Variable Importance and Interaction in Machine Learning via Genetic Algorithms

  • David Gabriel Guimarães,
  • Sabrina Sousa Carvalho,
  • Renato Higor do Nascimento,
  • Renato Tinós

摘要

This work explores the use of the Genetic Algorithm with Linkage Learning (GAwLL) for the feature selection problem. A notable byproduct of GAwLL application for feature selection is the generation of a variable interaction graph, offering insights into feature dependencies. To extend the original C++ implementation—which was limited to the K-nearest neighbors (KNN) algorithm—we introduce PyGAwLLfs, a Python-based framework designed to support feature selection for a wide range of machine learning models. Using PyGAwLLfs, we assess GAwLL performance across various models, including decision trees, random forests, and artificial neural networks. Furthermore, based on the internal mechanisms of GAwLL, we present a novel method for estimating variable importance in machine learning. Experimental evaluation on multiple datasets, including one from particle physics, demonstrates PyGAwLLfs effectiveness in the estimation of variable importance and interaction. We compare the variable importance scores generated by PyGAwLLfs with those produced by traditional approaches.