Scholarlink

<p>Despite advancements in Multi-Agent Deep Reinforcement Learning (MADRL), agents often exhibit fragility in dynamic adversarial environments due to reward function uncertainty and opponent policy shifts. Traditional RL algorithms struggle with unstable policy convergence and robustness issues under such uncertainties, as they inadequately model worst-case adversarial perturbations. To address this, we propose Robust-M3DDPG, a robust minimax multi-agent reinforcement learning framework that integrates Nash Equilibrium principles with minimax optimization. The approach formalizes the problem as a Robust Markov Game, explicitly modeling adversarial disturbances during policy optimization to enhance robustness in non-stationary environments. Key contributions include: (1) developing an Actor-Critic algorithm-based method to determine Nash Equilibrium policies; (2) extending the widely used Multi-Agent Deep Deterministic Policy Gradient Algorithm (MADDPG) with Robust Markov Games and minimax optimization for robust policy learning; and (3) proposing a Multi-Agent Adversarial Learning (MAAL) framework for efficiently solving adversarial policies to fulfill the minimax requirements. Evaluations across four cooperative-competitive multi-agent environments with different reward uncertainty levels demonstrate that Robust-M3DDPG significantly outperforms existing MADRL baselines in scenarios with high reward uncertainty and adversarial dynamics. This work bridges the gap between theoretical robustness guarantees and practical multi-agent reinforcement learning deployment under real-world perturbations.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Robust minimax multi-agent deep deterministic policy gradient for reward uncertainty

  • 期刊论文

  • 发表日期：

    2026年5月2日

查看全文
  • Daicheng Song,
  • Qiming Yang,
  • Yixuan Lin,
  • Haoxuan Zeng,
  • Jingwen Chong,
  • Li Song

摘要

Despite advancements in Multi-Agent Deep Reinforcement Learning (MADRL), agents often exhibit fragility in dynamic adversarial environments due to reward function uncertainty and opponent policy shifts. Traditional RL algorithms struggle with unstable policy convergence and robustness issues under such uncertainties, as they inadequately model worst-case adversarial perturbations. To address this, we propose Robust-M3DDPG, a robust minimax multi-agent reinforcement learning framework that integrates Nash Equilibrium principles with minimax optimization. The approach formalizes the problem as a Robust Markov Game, explicitly modeling adversarial disturbances during policy optimization to enhance robustness in non-stationary environments. Key contributions include: (1) developing an Actor-Critic algorithm-based method to determine Nash Equilibrium policies; (2) extending the widely used Multi-Agent Deep Deterministic Policy Gradient Algorithm (MADDPG) with Robust Markov Games and minimax optimization for robust policy learning; and (3) proposing a Multi-Agent Adversarial Learning (MAAL) framework for efficiently solving adversarial policies to fulfill the minimax requirements. Evaluations across four cooperative-competitive multi-agent environments with different reward uncertainty levels demonstrate that Robust-M3DDPG significantly outperforms existing MADRL baselines in scenarios with high reward uncertainty and adversarial dynamics. This work bridges the gap between theoretical robustness guarantees and practical multi-agent reinforcement learning deployment under real-world perturbations.