Increasing diversity in decision forests without compromising efficiency represents a challenge in machine learning. In this sense, the Proactive Forest algorithm has been proposed as an improvement of Random Forest, by introducing an adaptive scheme for attribute selection based on probabilities. However, the proactive scheme that uses this algorithm depends on the importance of the attributes to build new trees has a disadvantage. At a certain point, the probability of training trees with attributes of low predictive power can increase considerably, affecting the efficiency of the model. As a solution to this possible problem, a new variant is presented, called Proactive Frequency Forest, which incorporates two main components. These components focus on: (1) an initial weighted assignment of attribute selection probabilities, calculated using four statistical techniques, and (2) a dynamic updating strategy based on the frequency of occurrence of attributes during forest construction. Experimental validation, performed on multiple data sets, shows that the proposal significantly increases model diversity without negatively affecting predictive performance.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Proactive Frequency Forest: A Forest Construction Scheme Based on Proactive Forest

  • Javier García Hernández,
  • Nayma Cepero Pérez,
  • Daniel Pardo Echevarría

摘要

Increasing diversity in decision forests without compromising efficiency represents a challenge in machine learning. In this sense, the Proactive Forest algorithm has been proposed as an improvement of Random Forest, by introducing an adaptive scheme for attribute selection based on probabilities. However, the proactive scheme that uses this algorithm depends on the importance of the attributes to build new trees has a disadvantage. At a certain point, the probability of training trees with attributes of low predictive power can increase considerably, affecting the efficiency of the model. As a solution to this possible problem, a new variant is presented, called Proactive Frequency Forest, which incorporates two main components. These components focus on: (1) an initial weighted assignment of attribute selection probabilities, calculated using four statistical techniques, and (2) a dynamic updating strategy based on the frequency of occurrence of attributes during forest construction. Experimental validation, performed on multiple data sets, shows that the proposal significantly increases model diversity without negatively affecting predictive performance.