Scholarlink

The rapid advancement of intelligent cabin underscores the growing demand for radar-based in-vehicle monitoring (RIM) system with inherent contactless and non-intrusive characteristics. Meanwhile, the Convolutional vision Transformer (CvT) improves Vision Transformer (ViT) in performance and efficiency by introducing convolutions to leverage the advantages of both designs. However, existing RIM tasks such as occupancy detection, gesture recognition, drowsiness detection and vital sign monitoring typically use specialized approaches. As a result, there is no unified framework adaptable to diverse downstream applications. Moreover, integrating heterogeneous data modalities like audio to reinforce radar sensing remains challenging. This problem is especially pronounced when dense multi-path reflections occur in confined vehicle interiors. To address these challenges, we propose a novel RIM framework based on Convolutional vision Transformer (RIMCvT). By fine-tuning the pre-trained CvT model, the framework efficiently adapts to various tasks with limited labeled data while mitigating overfitting. RIMCvT also effectively integrates multiple data modalities through specially designed pre-processing pipelines. To further enhance the CvT model’s adaptability, a lightweight convolutional bypass module is introduced for fine-tuning within the feature space. This module allows the CvT model to efficiently transfer pre-trained knowledge to downstream tasks and maintain its generalization ability. We validate our approach on four task-specific datasets. Experimental results show that the proposed framework generalizes across different tasks. It matches or even outperforms existing state-of-the-art methods, achieving 93.60% accuracy in gesture recognition, 91.97% in drowsiness detection, 91.12% in vital sign monitoring and 95.04% in occupancy detection. These scores yield an average accuracy of 92.93%, which exceeds the SOTA methods by 10.8%. The results underscores RIMCvT’s ability to leverage information from radar and other signal (e.g. audio) across multiple tasks. The dataset and code are available at https://github.com/bupt-uwb/RIMCvT .

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

RIMCvT: Empowering Radar-Based In-vehicle Monitoring with Fine-Tuning Pre-trained Convolutional Vision Transformer

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Jiahang Guo,
  • Xikang Jiang,
  • Chong Rao,
  • Lin Zhang,
  • Lei Li

摘要

The rapid advancement of intelligent cabin underscores the growing demand for radar-based in-vehicle monitoring (RIM) system with inherent contactless and non-intrusive characteristics. Meanwhile, the Convolutional vision Transformer (CvT) improves Vision Transformer (ViT) in performance and efficiency by introducing convolutions to leverage the advantages of both designs. However, existing RIM tasks such as occupancy detection, gesture recognition, drowsiness detection and vital sign monitoring typically use specialized approaches. As a result, there is no unified framework adaptable to diverse downstream applications. Moreover, integrating heterogeneous data modalities like audio to reinforce radar sensing remains challenging. This problem is especially pronounced when dense multi-path reflections occur in confined vehicle interiors. To address these challenges, we propose a novel RIM framework based on Convolutional vision Transformer (RIMCvT). By fine-tuning the pre-trained CvT model, the framework efficiently adapts to various tasks with limited labeled data while mitigating overfitting. RIMCvT also effectively integrates multiple data modalities through specially designed pre-processing pipelines. To further enhance the CvT model’s adaptability, a lightweight convolutional bypass module is introduced for fine-tuning within the feature space. This module allows the CvT model to efficiently transfer pre-trained knowledge to downstream tasks and maintain its generalization ability. We validate our approach on four task-specific datasets. Experimental results show that the proposed framework generalizes across different tasks. It matches or even outperforms existing state-of-the-art methods, achieving 93.60% accuracy in gesture recognition, 91.97% in drowsiness detection, 91.12% in vital sign monitoring and 95.04% in occupancy detection. These scores yield an average accuracy of 92.93%, which exceeds the SOTA methods by 10.8%. The results underscores RIMCvT’s ability to leverage information from radar and other signal (e.g. audio) across multiple tasks. The dataset and code are available at https://github.com/bupt-uwb/RIMCvT .