Scholarlink

In recent years, both discrete-time reinforcement learning (DTRL) and continuous-time reinforcement learning (CTRL) have been successfully applied to various continuous control tasks. However, unlike DTRL, most studies in CTRL have focused primarily on algorithm improvements, with limited literature discussing extensive model parameter exploration. The lack of attention to model parameters poses significant challenges for algorithm reproducibility and performance tuning. In this paper, we present a large-scale experimental analysis of hyperparameter tuning in CTRL based on the Hamilton-Jacobi DQN (HJDQN) algorithm. Our objective is to improve the ability of CTRL so as to minimize performance variations caused by irreproducible and easily misunderstood behaviors, as well as the wastage of significant computational resources. Experimental results show that the four environments Hopper-v2, HalfCheetah-v2, Walker2d-v2, and Swimmer-v2 all have advantages when \(\gamma \) is larger, and the average returns are optimal when the sampling intervals h are set as 2.0, 0.2, 4.0, and 2.0 respectively.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Implementation Matters of Continuous-Time Deep Reinforcement Learning: A Case Study of HJDQN Algorithm

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Jin-Qiang Wang,
  • Yuanbo Jiang,
  • Lirong Song,
  • Lan Guo,
  • Binbin Yong,
  • Rui Zhou,
  • Xin Liu,
  • Qingguo Zhou

摘要

In recent years, both discrete-time reinforcement learning (DTRL) and continuous-time reinforcement learning (CTRL) have been successfully applied to various continuous control tasks. However, unlike DTRL, most studies in CTRL have focused primarily on algorithm improvements, with limited literature discussing extensive model parameter exploration. The lack of attention to model parameters poses significant challenges for algorithm reproducibility and performance tuning. In this paper, we present a large-scale experimental analysis of hyperparameter tuning in CTRL based on the Hamilton-Jacobi DQN (HJDQN) algorithm. Our objective is to improve the ability of CTRL so as to minimize performance variations caused by irreproducible and easily misunderstood behaviors, as well as the wastage of significant computational resources. Experimental results show that the four environments Hopper-v2, HalfCheetah-v2, Walker2d-v2, and Swimmer-v2 all have advantages when \(\gamma \) is larger, and the average returns are optimal when the sampling intervals h are set as 2.0, 0.2, 4.0, and 2.0 respectively.