Scholarlink

<p>Multi-agent reinforcement learning (MARL) has achieved substantial progress in cooperative decision-making, but learning remains difficult in environments with sparse rewards, long decision horizons, and strong inter-agent coupling. Existing methods usually optimize low-level policies directly from numerical observations, which can limit sample efficiency and make it difficult to incorporate structured strategic guidance. Here we propose LEHCA, a hierarchical MARL framework that uses a large language model as a coarse-timescale Commander to provide high-level semantic guidance for value-decomposition-based policy learning. The Commander receives structured textual summaries derived from observable environment information and generates strategic sub-goals, semantic reward-shaping rules, and action-level constraints. These outputs are grounded in low-level QMIX-based agents through two modular mechanisms: semantic reward shaping, which converts abstract sub-goals into dense auxiliary learning signals, and dynamic action masking, which guides exploration toward strategically relevant actions. Experiments on eight StarCraft multi-agent challenge scenarios show that LEHCA improves over QMIX across the evaluated maps in the reported metrics, with larger gains in heterogeneous, sparse-reward, and outnumbered settings. Additional comparisons with QPLEX, MAVEN, and MAPPO on representative scenarios indicate stronger early-stage learning efficiency, while ablation studies and non-LLM control variants show that both hierarchical guidance and LLM-generated semantic reasoning contribute to performance. A lightweight cooperative navigation experiment in the multi-agent particle environment further suggests that the framework can be instantiated beyond StarCraft micromanagement. These results support hierarchical LLM-guided MARL as a promising approach for improving learning efficiency, coordination, and interpretability in cooperative multi-agent systems.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A hierarchical multi-agent reinforcement learning framework with high-level guidance from large language models

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月25日

查看全文
  • Jinyin Bai,
  • Wei Zhu,
  • Xiangchen Wang,
  • KaiYang Kou,
  • Shiluo Guo,
  • Shuhong Liu,
  • Dong Li,
  • Tianjin Ni,
  • Jinji Zhou,
  • Yihao Zhong

摘要

Multi-agent reinforcement learning (MARL) has achieved substantial progress in cooperative decision-making, but learning remains difficult in environments with sparse rewards, long decision horizons, and strong inter-agent coupling. Existing methods usually optimize low-level policies directly from numerical observations, which can limit sample efficiency and make it difficult to incorporate structured strategic guidance. Here we propose LEHCA, a hierarchical MARL framework that uses a large language model as a coarse-timescale Commander to provide high-level semantic guidance for value-decomposition-based policy learning. The Commander receives structured textual summaries derived from observable environment information and generates strategic sub-goals, semantic reward-shaping rules, and action-level constraints. These outputs are grounded in low-level QMIX-based agents through two modular mechanisms: semantic reward shaping, which converts abstract sub-goals into dense auxiliary learning signals, and dynamic action masking, which guides exploration toward strategically relevant actions. Experiments on eight StarCraft multi-agent challenge scenarios show that LEHCA improves over QMIX across the evaluated maps in the reported metrics, with larger gains in heterogeneous, sparse-reward, and outnumbered settings. Additional comparisons with QPLEX, MAVEN, and MAPPO on representative scenarios indicate stronger early-stage learning efficiency, while ablation studies and non-LLM control variants show that both hierarchical guidance and LLM-generated semantic reasoning contribute to performance. A lightweight cooperative navigation experiment in the multi-agent particle environment further suggests that the framework can be instantiated beyond StarCraft micromanagement. These results support hierarchical LLM-guided MARL as a promising approach for improving learning efficiency, coordination, and interpretability in cooperative multi-agent systems.