Channel State Information (CSI) based sign language recognition technology provides an innovative solution to improve the involvement of hearing impaired people in education and employment. However, its practical application still encounters two key challenges: (I) Existing systems mostly operate effectively only within the source domain; (II) Cross-domain recognition requires extensive pretraining on target domain data, which limits the model’s generalization capability. Moreover, the complex training strategies further raise the threshold for model deployment. To address the above problems, this paper proposes CSI-Cro: A cross-domain CSI sign language recognition system based on Dual-Attention Feature Decoupled Network. CSI-Cro consists of a signal preprocessing module and a recognition module: The preprocessing module converts the raw CSI signal into a high-dimensional spectrogram via time-frequency transforms. The recognition module innovatively employs a feature decoupling mechanism, using parallel domain and sign language feature extractors to decouple the mixed feature into domain-independent sign language representations and domain-dependent representations. An adaptive feature reconstruction layer then generates a highly discriminative fused tensor, ultimately achieving precise sign language classification. The experimental results show that CSI-Cro is able to achieve 96.85% in-domain recognition accuracy, and 92.73% and 91.3% accuracy in cross-position and cross-user test experiments, respectively, which is better than the current CSI cross-domain recognition system.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

CSI-Cro: A Cross-Domain Sign Language Recognition System Based on Dual-Attention Feature Decoupled Network

  • Junru Zheng,
  • Sihan Liang,
  • Wenlin Li,
  • Gaofei Sun,
  • Xiaobing Xian

摘要

Channel State Information (CSI) based sign language recognition technology provides an innovative solution to improve the involvement of hearing impaired people in education and employment. However, its practical application still encounters two key challenges: (I) Existing systems mostly operate effectively only within the source domain; (II) Cross-domain recognition requires extensive pretraining on target domain data, which limits the model’s generalization capability. Moreover, the complex training strategies further raise the threshold for model deployment. To address the above problems, this paper proposes CSI-Cro: A cross-domain CSI sign language recognition system based on Dual-Attention Feature Decoupled Network. CSI-Cro consists of a signal preprocessing module and a recognition module: The preprocessing module converts the raw CSI signal into a high-dimensional spectrogram via time-frequency transforms. The recognition module innovatively employs a feature decoupling mechanism, using parallel domain and sign language feature extractors to decouple the mixed feature into domain-independent sign language representations and domain-dependent representations. An adaptive feature reconstruction layer then generates a highly discriminative fused tensor, ultimately achieving precise sign language classification. The experimental results show that CSI-Cro is able to achieve 96.85% in-domain recognition accuracy, and 92.73% and 91.3% accuracy in cross-position and cross-user test experiments, respectively, which is better than the current CSI cross-domain recognition system.