Gated Multi-modal Fusion for Vision-Based Locomotion Control of Quadruped Robots
摘要
Recent advances in deep reinforcement learning (DRL) have significantly improved quadruped locomotion in complex environments. Traditional blind policies, relying only on proprioception, are robust and transferable but lack environmental awareness, reducing effectiveness on unseen terrains. Vision-based approaches provide foresight but often suffer from poor modality fusion, inefficient feature use, and sensitivity to noise. To address these issues, we propose a multimodal control framework that integrates visual and proprioceptive inputs, combining the strengths of both approaches. A novel gated fusion module enables adaptive, fine-grained interaction between modalities, dynamically adjusting fusion based on the current state to reduce noise and improve robustness. Benchmark results across diverse terrains show our method consistently outperforms baselines in effectiveness and generalization.