In the development of computer vision, The effectiveness of traditional unimodal perception is constrained in complex environments. Such adverse conditions significantly challenge its capability for reliable target recognition. Multimodal perception fuses complementary sensor data (e.g., RGB images, depth maps, infrared images, LIDAR point clouds), which significantly improves perceptual robustness and understanding. In this paper, we focus on the application of multimodal perception in target recognition, and systematically review the recent advances at home and abroad. Based on modal combinations and application scenarios, the mainstream methods are categorized into RGB-D fusion, infrared-visible fusion, LIDAR-vision fusion, and other major categories, and the representative technologies are compared. Then, the core challenges faced by the current system are analyzed in depth: modal heterogeneity, cross-modal alignment difficulty, modal missing robustness and computational resource consumption. The trends of emerging techniques such as Transformer-based, self-supervised learning, and cross-modal unified modeling and their potentials to address the challenges are also explored. The aim of this paper is to provide a clear and systematic reference framework for researchers in this field, and to provide theoretical and practical references for subsequent research.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Application of Multimodal Perception in Target Recognition

  • Jiaxin Tang,
  • Lichuan Ning,
  • Yuanmin Xie

摘要

In the development of computer vision, The effectiveness of traditional unimodal perception is constrained in complex environments. Such adverse conditions significantly challenge its capability for reliable target recognition. Multimodal perception fuses complementary sensor data (e.g., RGB images, depth maps, infrared images, LIDAR point clouds), which significantly improves perceptual robustness and understanding. In this paper, we focus on the application of multimodal perception in target recognition, and systematically review the recent advances at home and abroad. Based on modal combinations and application scenarios, the mainstream methods are categorized into RGB-D fusion, infrared-visible fusion, LIDAR-vision fusion, and other major categories, and the representative technologies are compared. Then, the core challenges faced by the current system are analyzed in depth: modal heterogeneity, cross-modal alignment difficulty, modal missing robustness and computational resource consumption. The trends of emerging techniques such as Transformer-based, self-supervised learning, and cross-modal unified modeling and their potentials to address the challenges are also explored. The aim of this paper is to provide a clear and systematic reference framework for researchers in this field, and to provide theoretical and practical references for subsequent research.