Scholarlink

<p>In unstructured environments, robotic manipulation tasks often require the integration of visual and tactile feedback, yet learning effective strategies directly from environmental interactions remains challenging. While Reinforcement Learning (RL) excels in optimizing control policies for high-dimensional inputs, redundant state representations and low sample efficiency limit its deployment in real-world robotic systems. This paper proposes an Adaptive Multimodal Fusion (AMF) Transformer for high-precision Peg-in-Hole (PiH) assembly, which performs cross-modal representation of visual and force cues. The AMF includes a feature interaction module and an adaptive gating module uncovering latent relationships between different modalities, providing a dynamic multimodal fusion method and high-quality operational data for the robot. Experimental results demonstrate that our multimodal representation improves manipulation flexibility and robustness, enabling zero-shot transfer of the learned model to real robots. On two previously unseen peg-hole pairs, the system achieves over 95% insertion success at 0.1&#xa0;mm clearance.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Adaptive multimodal fusion-driven reinforcement learning for robotic peg-in-hole assembly

  • 期刊论文

  • 发表日期：

    2026年3月23日

查看全文
  • Jiqiang Qu,
  • Dexue Bi,
  • Yiding Liu

摘要

In unstructured environments, robotic manipulation tasks often require the integration of visual and tactile feedback, yet learning effective strategies directly from environmental interactions remains challenging. While Reinforcement Learning (RL) excels in optimizing control policies for high-dimensional inputs, redundant state representations and low sample efficiency limit its deployment in real-world robotic systems. This paper proposes an Adaptive Multimodal Fusion (AMF) Transformer for high-precision Peg-in-Hole (PiH) assembly, which performs cross-modal representation of visual and force cues. The AMF includes a feature interaction module and an adaptive gating module uncovering latent relationships between different modalities, providing a dynamic multimodal fusion method and high-quality operational data for the robot. Experimental results demonstrate that our multimodal representation improves manipulation flexibility and robustness, enabling zero-shot transfer of the learned model to real robots. On two previously unseen peg-hole pairs, the system achieves over 95% insertion success at 0.1 mm clearance.