Scholarlink

<p>Offline reinforcement learning (RL) allows agents to learn effective policies without any direct interaction with the environment by relying solely on pre-collected data. However, conventional offline RL methods face challenges such as out-of-distribution generalization and the curse of dimensionality. To mitigate these difficulties, we propose the convex Markov game-based Multi-Agent Decision Transformer (CovMADT) algorithm, which provides a theoretical guarantee of a pure-strategy Nash equilibrium under strictly concave utility functions. Within the framework of convex Markov games (cMG), we estimate state transition probabilities in continuous multi-dimensional spaces by leveraging reproducing kernel Hilbert space (RKHS) embeddings and the empirical distribution of states, while simultaneously modeling imitation-like utilities. To improve model performance, we integrate Mean-Field Value Iteration (MFVI) as the critic, exploit permutation invariance and kernel techniques to optimize computational efficiency, and empirically validate their effectiveness in mitigating Transformer degradation. The experimental results demonstrate that CovMADT enables agents to learn complex coordination strategies and achieve superior performance in both competitive and cooperative tasks by accurately capturing underlying physical dynamics. The code will be published after it is accepted. Within the framework of convex Markov games (cMG), we estimate state transition probabilities in continuous multi-dimensional spaces by leveraging reproducing kernel Hilbert space (RKHS) embeddings and the empirical distribution of states, while simultaneously modeling imitation-regularized utilities. To improve model performance, we integrate Mean-Field Value Iteration (MFVI) as the critic, exploit permutation invariance and kernel techniques to optimize computational efficiency, and empirically validate their effectiveness in mitigating Transformer degradation. The experimental results demonstrate that CovMADT enables agents to learn complex coordination strategies and achieve superior performance in both competitive and cooperative tasks by accurately capturing underlying physical dynamics. The code will be published after it is accepted.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

CovMADT: efficient offline multi-agent reinforcement learning via convex Markov games

  • 期刊论文

  • 发表日期：

    2026年4月30日

查看全文
  • Sikong Wen,
  • Rui Wang,
  • Dechen Wu,
  • Zixuan Wang

摘要

Offline reinforcement learning (RL) allows agents to learn effective policies without any direct interaction with the environment by relying solely on pre-collected data. However, conventional offline RL methods face challenges such as out-of-distribution generalization and the curse of dimensionality. To mitigate these difficulties, we propose the convex Markov game-based Multi-Agent Decision Transformer (CovMADT) algorithm, which provides a theoretical guarantee of a pure-strategy Nash equilibrium under strictly concave utility functions. Within the framework of convex Markov games (cMG), we estimate state transition probabilities in continuous multi-dimensional spaces by leveraging reproducing kernel Hilbert space (RKHS) embeddings and the empirical distribution of states, while simultaneously modeling imitation-like utilities. To improve model performance, we integrate Mean-Field Value Iteration (MFVI) as the critic, exploit permutation invariance and kernel techniques to optimize computational efficiency, and empirically validate their effectiveness in mitigating Transformer degradation. The experimental results demonstrate that CovMADT enables agents to learn complex coordination strategies and achieve superior performance in both competitive and cooperative tasks by accurately capturing underlying physical dynamics. The code will be published after it is accepted. Within the framework of convex Markov games (cMG), we estimate state transition probabilities in continuous multi-dimensional spaces by leveraging reproducing kernel Hilbert space (RKHS) embeddings and the empirical distribution of states, while simultaneously modeling imitation-regularized utilities. To improve model performance, we integrate Mean-Field Value Iteration (MFVI) as the critic, exploit permutation invariance and kernel techniques to optimize computational efficiency, and empirically validate their effectiveness in mitigating Transformer degradation. The experimental results demonstrate that CovMADT enables agents to learn complex coordination strategies and achieve superior performance in both competitive and cooperative tasks by accurately capturing underlying physical dynamics. The code will be published after it is accepted.