<p>Hand-gesture-based sign language recognition (SLR) is one of the most advanced applications of machine learning and computer vision for recognizing hand gestures. While automatic recognition systems have been developed for languages such as English, Turkish, Arabic, and others, challenges remain in achieving standardized recognition of Bangla Sign Language (BSL). Despite significant progress by researchers in addressing BSL recognition issues, several challenges remain, particularly in skeleton- and transformer-based approaches. Moreover, the lack of evaluation of BSL models under diverse environmental conditions limits the generalizability of existing systems in real-world scenarios. Current BSL recognition systems offer limited insights into their generalization capabilities, as they are often tested on datasets with a small number of BSL alphabets, large variations in gestures, and easily distinguishable signs. To address these limitations, we propose a spatiotemporal attention-based BSL recognition model that leverages hand joint skeletons extracted from image sequences. The primary goal of using hand-skeleton-based BSL data is to ensure privacy, reduce computational costs, and minimize hardware requirements, using low-resolution image sequences. Our model captures discriminative structural displacements and short-range dependencies by projecting unified joint features into a high-dimensional feature space. Specifically, the combination of a Separable TCN and a multi-head spatiotemporal attention architecture yields high accuracy. Extensive experiments on the proposed dataset, as well as two benchmark BSL datasets, across various evaluation settings (e.g., intra- and inter-dataset), demonstrate that our model achieves competitive performance with significantly reduced computational complexity, outperforming existing systems in speed.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Bengali Sign Language Recognition Through Hand Pose Estimation Using Multi-branch Spatial-Temporal Attention Model

  • Abu Saleh Musa Miah,
  • Md. Al Mehedi Hasan,
  • Md Hadiuzzaman,
  • Muhammad Nazrul Islam,
  • Jungpil Shin

摘要

Hand-gesture-based sign language recognition (SLR) is one of the most advanced applications of machine learning and computer vision for recognizing hand gestures. While automatic recognition systems have been developed for languages such as English, Turkish, Arabic, and others, challenges remain in achieving standardized recognition of Bangla Sign Language (BSL). Despite significant progress by researchers in addressing BSL recognition issues, several challenges remain, particularly in skeleton- and transformer-based approaches. Moreover, the lack of evaluation of BSL models under diverse environmental conditions limits the generalizability of existing systems in real-world scenarios. Current BSL recognition systems offer limited insights into their generalization capabilities, as they are often tested on datasets with a small number of BSL alphabets, large variations in gestures, and easily distinguishable signs. To address these limitations, we propose a spatiotemporal attention-based BSL recognition model that leverages hand joint skeletons extracted from image sequences. The primary goal of using hand-skeleton-based BSL data is to ensure privacy, reduce computational costs, and minimize hardware requirements, using low-resolution image sequences. Our model captures discriminative structural displacements and short-range dependencies by projecting unified joint features into a high-dimensional feature space. Specifically, the combination of a Separable TCN and a multi-head spatiotemporal attention architecture yields high accuracy. Extensive experiments on the proposed dataset, as well as two benchmark BSL datasets, across various evaluation settings (e.g., intra- and inter-dataset), demonstrate that our model achieves competitive performance with significantly reduced computational complexity, outperforming existing systems in speed.