Scholarlink

<p>Transformers show great potential in offline reinforcement learning via trajectory sequence modeling for action prediction. However, existing Transformer-based methods face limitations, such as ineffective trajectory stitching and the neglect of deep interactions within and between multimodal information in trajectories. We propose CGM, an offline reinforcement learning approach that combines Generalized Advantage Estimation with Modality Decomposition Interaction (MDI) to address these challenges. Generalized Advantage Estimation relabels the dataset to enhance trajectory stitching effectiveness. MDI consists of an encoder and a decoder. The encoder integrates an intra-modal interaction mechanism based on ConvFormer and an inter-modal interaction mechanism based on a dual-Transformer architecture to enable information exchange within and across modalities. In intra-modal interaction, the convolutional properties of ConvFormer effectively capture the associative information within respective modalities of states and actions. In inter-modal interaction, the dual-Transformer architecture facilitates multimodal information exchange for states and actions separately, fully exploring potential correlations between different modal data to achieve deep cross-modal information interaction. The decoder utilizes advantage values to optimize action prediction. We compared CGM with state-of-the-art baseline methods on the D4RL dataset. On the MuJoCo dataset, our proposed method outperforms the optimal comparison method by 2.89% in performance.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Offline reinforcement learning combining generalized advantage estimation and modality decomposition interaction

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年5月4日

查看全文
  • Kaixin Jin,
  • Lifang Wang,
  • Xiwen Wang,
  • Wei Guo,
  • Qiang Han,
  • Xiaoqing Yu

摘要

Transformers show great potential in offline reinforcement learning via trajectory sequence modeling for action prediction. However, existing Transformer-based methods face limitations, such as ineffective trajectory stitching and the neglect of deep interactions within and between multimodal information in trajectories. We propose CGM, an offline reinforcement learning approach that combines Generalized Advantage Estimation with Modality Decomposition Interaction (MDI) to address these challenges. Generalized Advantage Estimation relabels the dataset to enhance trajectory stitching effectiveness. MDI consists of an encoder and a decoder. The encoder integrates an intra-modal interaction mechanism based on ConvFormer and an inter-modal interaction mechanism based on a dual-Transformer architecture to enable information exchange within and across modalities. In intra-modal interaction, the convolutional properties of ConvFormer effectively capture the associative information within respective modalities of states and actions. In inter-modal interaction, the dual-Transformer architecture facilitates multimodal information exchange for states and actions separately, fully exploring potential correlations between different modal data to achieve deep cross-modal information interaction. The decoder utilizes advantage values to optimize action prediction. We compared CGM with state-of-the-art baseline methods on the D4RL dataset. On the MuJoCo dataset, our proposed method outperforms the optimal comparison method by 2.89% in performance.