Scholarlink

<p>A long-running challenge in the reinforcement learning (RL) community has been to train a goal-conditioned agent in sparse reward environment such that it also generalizes to unseen goals. We propose a novel goal-conditioned RL algorithm; Multi-Teacher Asymmetric Self-Play, which allows multiple agents (i.e., the <i>teachers</i>) to create a successful curriculum for another agent (i.e., the <i>student</i>) and empirically demonstrate its effectiveness on complex domains like FetchReach and a novel driving simulator designed for goal-conditioned RL. Our results show a 30-40% improvement over the baseline while also improving the learning speed of the student. We attribute this improvement in performance to the better exploration and coverage of the state space by multiple teacher agents. In addition, the results show that completely new students can learn offline from the goals generated by teachers trained with a previous student, reducing the computational cost by around 95%. This is crucial in the context of application domains where repeatedly training a teacher agent is expensive or even infeasible.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Do as you teach: a multi-teacher approach to self-play in deep reinforcement learning

  • 期刊论文

  • 发表日期：

    2025年1月7日

查看全文
  • Chaitanya Kharyal,
  • Sai Krishna Gottipati,
  • Tanmay Kumar Sinha,
  • Fatemeh Abdollahi,
  • Srijita Das,
  • Matthew E. Taylor

摘要

A long-running challenge in the reinforcement learning (RL) community has been to train a goal-conditioned agent in sparse reward environment such that it also generalizes to unseen goals. We propose a novel goal-conditioned RL algorithm; Multi-Teacher Asymmetric Self-Play, which allows multiple agents (i.e., the teachers) to create a successful curriculum for another agent (i.e., the student) and empirically demonstrate its effectiveness on complex domains like FetchReach and a novel driving simulator designed for goal-conditioned RL. Our results show a 30-40% improvement over the baseline while also improving the learning speed of the student. We attribute this improvement in performance to the better exploration and coverage of the state space by multiple teacher agents. In addition, the results show that completely new students can learn offline from the goals generated by teachers trained with a previous student, reducing the computational cost by around 95%. This is crucial in the context of application domains where repeatedly training a teacher agent is expensive or even infeasible.