<p>Reinforcement Learning (RL)-based programs’ quality often falls short of software engineering industry standards. This maybe due to the complexity of RL algorithms or their black-box nature, which isolates them from the rest of the system. There is a need to close the quality gap, as the accumulated technical debt for RL-based programs continues to grow. To achieve this goal, the use of analysis tools may provide a first-hand view of problematic regions of the code base, and suggest possible improvements. This paper, presents VAR Check, a tool to Visualize an Abstract Representation of programs, as a first step in the quality analysis of RL programs. VAR Check provides a Voronoi representation of programs that visualizes areas where code smells are present. Furthermore, the tool can be used to detect divergence between different program implementations to a solution, or similarities across programs from multiple applications. Such information might be useful to identify performance leaks, or to create abstractions common to multiple solutions. We evaluate VAR Check, on a corpus of 97 RL-based programs divided into six benchmarks covering application of tabular Q-learning and Deep Q-learning. VAR Check effectively identifies code smells, highlights implementation differences within application domains, detects faulty or underperforming variants, and uncovers cross-domain similarities, suggesting a standardized specification for RL programs.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

VAR Check: Quality Analysis of Reinforcement Learning Programs Using Voronoi Diagrams

  • Laura Rodriguez,
  • Ivana Dusparic,
  • Nicolás Cardozo

摘要

Reinforcement Learning (RL)-based programs’ quality often falls short of software engineering industry standards. This maybe due to the complexity of RL algorithms or their black-box nature, which isolates them from the rest of the system. There is a need to close the quality gap, as the accumulated technical debt for RL-based programs continues to grow. To achieve this goal, the use of analysis tools may provide a first-hand view of problematic regions of the code base, and suggest possible improvements. This paper, presents VAR Check, a tool to Visualize an Abstract Representation of programs, as a first step in the quality analysis of RL programs. VAR Check provides a Voronoi representation of programs that visualizes areas where code smells are present. Furthermore, the tool can be used to detect divergence between different program implementations to a solution, or similarities across programs from multiple applications. Such information might be useful to identify performance leaks, or to create abstractions common to multiple solutions. We evaluate VAR Check, on a corpus of 97 RL-based programs divided into six benchmarks covering application of tabular Q-learning and Deep Q-learning. VAR Check effectively identifies code smells, highlights implementation differences within application domains, detects faulty or underperforming variants, and uncovers cross-domain similarities, suggesting a standardized specification for RL programs.