<p>Feature selection (FS) remains a critical challenge in data mining for high-dimensional data with few instances, especially in the field of biomedical. Although ant colony optimization algorithm(ACO) has achieved good performance in this topic, which faces tough challenges in the setting of heuristic factor (HF), updating mechanisms of pheromone and computational scalability for ultra-high-dimensional data. To alleviate these issues, this paper proposes an improved ACO algorithm based on the self-attention mechanism (SAMACO), where the feature importance weight scores calculated by the self-attention mechanism are used as the HF for ACO. The improved probabilistic selection mechanism incorporates the idea of the roulette wheel selection, optimizing the selection process according to feature dimensions to control the number of feature subsets. The dynamic updating mechanism of pheromone integrates the elite strategy, as the number of iterations increases, the intensity of updating pheromone for ants with fitness in the top 10% also increases. Finally, the superiority of SAMACO is verified on 13 high-dimensional datasets with few instances in the biomedical field. Compared with the average performance of 9 representative algorithms such as GOOSE, NPDOA and ACO-RNN, SAMACO maintains the advantage in accuracy while reducing the feature dimension by 90.6% and improving computational efficiency by 75.3%, which indicates that SAMACO provides a potential solution to FS for high-dimensional data with few instances.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

SAMACO_FS: feature selection for high-dimensional few instances using ant colony optimization algorithm and self-attention mechanism

  • Zhiwei Ye,
  • An Song,
  • Huazhong Jin,
  • Wen Zhou,
  • Ting Cai,
  • Mingwei Wang,
  • Mengqing Mei,
  • Qiyi He,
  • Xiaochun Cheng

摘要

Feature selection (FS) remains a critical challenge in data mining for high-dimensional data with few instances, especially in the field of biomedical. Although ant colony optimization algorithm(ACO) has achieved good performance in this topic, which faces tough challenges in the setting of heuristic factor (HF), updating mechanisms of pheromone and computational scalability for ultra-high-dimensional data. To alleviate these issues, this paper proposes an improved ACO algorithm based on the self-attention mechanism (SAMACO), where the feature importance weight scores calculated by the self-attention mechanism are used as the HF for ACO. The improved probabilistic selection mechanism incorporates the idea of the roulette wheel selection, optimizing the selection process according to feature dimensions to control the number of feature subsets. The dynamic updating mechanism of pheromone integrates the elite strategy, as the number of iterations increases, the intensity of updating pheromone for ants with fitness in the top 10% also increases. Finally, the superiority of SAMACO is verified on 13 high-dimensional datasets with few instances in the biomedical field. Compared with the average performance of 9 representative algorithms such as GOOSE, NPDOA and ACO-RNN, SAMACO maintains the advantage in accuracy while reducing the feature dimension by 90.6% and improving computational efficiency by 75.3%, which indicates that SAMACO provides a potential solution to FS for high-dimensional data with few instances.