Sign language recognition (SLR) has gained increasing attention as a means of facilitating communication accessibility for the deaf and hard-of-hearing communities. This paper presents a comprehensive evaluation framework for isolated sign recognition, leveraging a high-dimensional spatiotemporal dataset composed of over 120,000 annotated video samples. Each sample is represented as a sequence of structured motion descriptors encoding dynamic hand gestures, facial cues, and upper-body pose information, enabling fine-grained modeling of temporal dynamics. We investigate and compare both traditional machine learning and deep learning models tailored for spatiotemporal gesture classification. Specifically, we evaluate (1) two classical machine learning classifiers—Random Forest and XGBoost—applied on pose-based feature representations, and (2) two deep neural network architectures: (i) Inflated 3D ConvNets (I3D), and (ii) a sequential LSTM + Dense model that processes keypoint sequences through recurrent and fully connected layers. Among all evaluated models, the LSTM + Dense architecture achieved the highest accuracy, the model achieved a final training accuracy of (92.92%) and a validation accuracy of (88.22%), confirming the effectiveness of recurrent-based temporal modeling in isolated sign classification tasks. The comparative results highlight the strengths and limitations of each approach and provide a robust foundation for future work on continuous sign recognition and the integration of multimodal learning strategies.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Isolated Sign Recognition from Pose Sequences: An Empirical Evaluation of ML and DL Models

  • Basma Lahmimsi,
  • Kenza Bengoud,
  • Nouhaila Lançar,
  • Karima Khouya

摘要

Sign language recognition (SLR) has gained increasing attention as a means of facilitating communication accessibility for the deaf and hard-of-hearing communities. This paper presents a comprehensive evaluation framework for isolated sign recognition, leveraging a high-dimensional spatiotemporal dataset composed of over 120,000 annotated video samples. Each sample is represented as a sequence of structured motion descriptors encoding dynamic hand gestures, facial cues, and upper-body pose information, enabling fine-grained modeling of temporal dynamics. We investigate and compare both traditional machine learning and deep learning models tailored for spatiotemporal gesture classification. Specifically, we evaluate (1) two classical machine learning classifiers—Random Forest and XGBoost—applied on pose-based feature representations, and (2) two deep neural network architectures: (i) Inflated 3D ConvNets (I3D), and (ii) a sequential LSTM + Dense model that processes keypoint sequences through recurrent and fully connected layers. Among all evaluated models, the LSTM + Dense architecture achieved the highest accuracy, the model achieved a final training accuracy of (92.92%) and a validation accuracy of (88.22%), confirming the effectiveness of recurrent-based temporal modeling in isolated sign classification tasks. The comparative results highlight the strengths and limitations of each approach and provide a robust foundation for future work on continuous sign recognition and the integration of multimodal learning strategies.