Scholarlink

<p>Sign language recognition (SLR) plays a critical role in bridging communication gaps for the hearing- and speech-impaired community. However, current systems struggle with challenges such as redundant video frames, dynamic backgrounds, signer variations, and limited real-time accuracy. Motivated by the need for an efficient, scalable, and accurate solution, this study proposes a novel deep learning framework for robust keyframe extraction and SLR. The method begins with the preparation of two benchmark datasets, INCLUDE and INCLUDE-50, followed by action-based keyframe extraction using structural similarity (SSIM), cosine similarity, and MSE/MAE thresholds. A customized CNN model named SignKeyNet-AR is developed, integrating atrous convolutions, residual connections, and a ResNet-50 backbone to improve gesture classification. The use of IGMM for background subtraction enhances data quality, while Optical Flow aids normalization. Experimental results show superior performance across multiple metrics (accuracy: 99.05%, mAP: 97.92%), outperforming state-of-the-art KFE techniques. This approach provides an effective, real-time solution for accurate sign language video recognition.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A novel key frame extraction using a deep learning model for sign language recognition on videos

  • 期刊论文

  • 发表日期：

    2025年10月20日

查看全文
  • M. Navyasri,
  • G. Jaya Suma

摘要

Sign language recognition (SLR) plays a critical role in bridging communication gaps for the hearing- and speech-impaired community. However, current systems struggle with challenges such as redundant video frames, dynamic backgrounds, signer variations, and limited real-time accuracy. Motivated by the need for an efficient, scalable, and accurate solution, this study proposes a novel deep learning framework for robust keyframe extraction and SLR. The method begins with the preparation of two benchmark datasets, INCLUDE and INCLUDE-50, followed by action-based keyframe extraction using structural similarity (SSIM), cosine similarity, and MSE/MAE thresholds. A customized CNN model named SignKeyNet-AR is developed, integrating atrous convolutions, residual connections, and a ResNet-50 backbone to improve gesture classification. The use of IGMM for background subtraction enhances data quality, while Optical Flow aids normalization. Experimental results show superior performance across multiple metrics (accuracy: 99.05%, mAP: 97.92%), outperforming state-of-the-art KFE techniques. This approach provides an effective, real-time solution for accurate sign language video recognition.