Decision trees (DTs) are explainable machine learning techniques applicable to classification and regression problems. Traditionally, DTs are built through a top-down greedy search, which is usually fast but may lead to sub-optimal solutions. An alternative approach involves the use of evolutionary algorithms (EAs), which allow for more global exploration that can yield simpler and accurate DTs. However, the EA-based DT induction is computationally demanding, especially for large-scale data. To alleviate these high computing requirements, various parallel and distributed accelerations are continuously explored. In this paper, we focus on a GPU-supported solution and extend it by improving memory management. To reduce synchronization points between GPU threads (both within and across blocks), the new solution avoids atomic functions. However, such an approach requires allocating individual result buffers for each thread, which increases memory requirements. To compensate for this, a compact in-memory representation of DTs is additionally applied. Moreover, an additional level of reduction is necessary. Experimental validation on various datasets, both artificial and real-life, shows that the enhanced solution further accelerates the EA-based DT induction. The results also reveal that the time savings increase as the dataset size grows and are influenced by DT size. Therefore, the hybrid solution provides the best time results, by applying synchronization avoidance for smaller DTs and the opposite strategy for larger ones.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Improved GPU Memory Management in Evolutionary Decision Tree Induction for Large-Scale Data

  • Krzysztof Jurczuk,
  • Daniel Reska,
  • Marcin Czajkowski,
  • Marek Kretowski

摘要

Decision trees (DTs) are explainable machine learning techniques applicable to classification and regression problems. Traditionally, DTs are built through a top-down greedy search, which is usually fast but may lead to sub-optimal solutions. An alternative approach involves the use of evolutionary algorithms (EAs), which allow for more global exploration that can yield simpler and accurate DTs. However, the EA-based DT induction is computationally demanding, especially for large-scale data. To alleviate these high computing requirements, various parallel and distributed accelerations are continuously explored. In this paper, we focus on a GPU-supported solution and extend it by improving memory management. To reduce synchronization points between GPU threads (both within and across blocks), the new solution avoids atomic functions. However, such an approach requires allocating individual result buffers for each thread, which increases memory requirements. To compensate for this, a compact in-memory representation of DTs is additionally applied. Moreover, an additional level of reduction is necessary. Experimental validation on various datasets, both artificial and real-life, shows that the enhanced solution further accelerates the EA-based DT induction. The results also reveal that the time savings increase as the dataset size grows and are influenced by DT size. Therefore, the hybrid solution provides the best time results, by applying synchronization avoidance for smaller DTs and the opposite strategy for larger ones.