Scholarlink

<p>In high-dimensional real-time strategy (RTS) environments with sparse rewards, PPO-style reinforcement learning can exhibit high training variance and exploration stagnation. Large language models (LLMs) provide semantic priors, yet naïve prompt injection may cause semantic–action mismatch and latency-induced control jitter. We propose PPO-Dynamic, a macro–meso–micro prompting framework that generates hierarchical instructions online and integrates them into PPO through state-aligned fusion. Prompts at all levels are mapped into a shared embedding space via the same encoder–projection pipeline and aligned to policy state features using cross-attention; a probabilistic gating strategy then regulates when semantic guidance should influence action selection to preserve stability. On the StarCraft II CollectMineralShards benchmark, PPO-Dynamic improves per-step collection efficiency (median 0.43 vs 0.28/0.27 minerals/step) and increases exploration coverage (35.87% vs 13.59%/15.93%), with gains that are statistically significant (two-sample Welch’s test, <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\varvec{p&lt;0.001}\)</EquationSource> </InlineEquation>). Because recent end-to-end LLM–SC2 baselines differ in task definitions and interaction protocols, strict numerical alignment is left to future unified benchmarking; we instead position our method as a controllable and reproducible integration mechanism for semantic guidance in real-time RL. Limitations include validation on a single collection benchmark and non-negligible local LLM overhead (<InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(\varvec{\approx 0.8}\)</EquationSource> </InlineEquation>&#xa0;s per 120 steps; <InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(\varvec{\approx 15\%}\)</EquationSource> </InlineEquation> wall-clock increase). Future work will extend the framework to broader RTS objectives and multi-agent coordination settings under standardized evaluation.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A reinforcement learning decision-making method with multi-granularity semantic prompts from large language models

  • 期刊论文

  • 发表日期：

    2026年5月19日

查看全文
  • Shilin Hao,
  • Gang Liu,
  • Xiaotian Guo,
  • Xiong Liu,
  • Dong Huang,
  • Wu Li

摘要

In high-dimensional real-time strategy (RTS) environments with sparse rewards, PPO-style reinforcement learning can exhibit high training variance and exploration stagnation. Large language models (LLMs) provide semantic priors, yet naïve prompt injection may cause semantic–action mismatch and latency-induced control jitter. We propose PPO-Dynamic, a macro–meso–micro prompting framework that generates hierarchical instructions online and integrates them into PPO through state-aligned fusion. Prompts at all levels are mapped into a shared embedding space via the same encoder–projection pipeline and aligned to policy state features using cross-attention; a probabilistic gating strategy then regulates when semantic guidance should influence action selection to preserve stability. On the StarCraft II CollectMineralShards benchmark, PPO-Dynamic improves per-step collection efficiency (median 0.43 vs 0.28/0.27 minerals/step) and increases exploration coverage (35.87% vs 13.59%/15.93%), with gains that are statistically significant (two-sample Welch’s test, \(\varvec{p<0.001}\) ). Because recent end-to-end LLM–SC2 baselines differ in task definitions and interaction protocols, strict numerical alignment is left to future unified benchmarking; we instead position our method as a controllable and reproducible integration mechanism for semantic guidance in real-time RL. Limitations include validation on a single collection benchmark and non-negligible local LLM overhead ( \(\varvec{\approx 0.8}\)  s per 120 steps; \(\varvec{\approx 15\%}\) wall-clock increase). Future work will extend the framework to broader RTS objectives and multi-agent coordination settings under standardized evaluation.