Isolated Sign Recognition from Pose Sequences: An Empirical Evaluation of ML and DL Models
摘要
Sign language recognition (SLR) has gained increasing attention as a means of facilitating communication accessibility for the deaf and hard-of-hearing communities. This paper presents a comprehensive evaluation framework for isolated sign recognition, leveraging a high-dimensional spatiotemporal dataset composed of over 120,000 annotated video samples. Each sample is represented as a sequence of structured motion descriptors encoding dynamic hand gestures, facial cues, and upper-body pose information, enabling fine-grained modeling of temporal dynamics. We investigate and compare both traditional machine learning and deep learning models tailored for spatiotemporal gesture classification. Specifically, we evaluate (1) two classical machine learning classifiers—Random Forest and XGBoost—applied on pose-based feature representations, and (2) two deep neural network architectures: (i) Inflated 3D ConvNets (I3D), and (ii) a sequential LSTM + Dense model that processes keypoint sequences through recurrent and fully connected layers. Among all evaluated models, the LSTM + Dense architecture achieved the highest accuracy, the model achieved a final training accuracy of (92.92%) and a validation accuracy of (88.22%), confirming the effectiveness of recurrent-based temporal modeling in isolated sign classification tasks. The comparative results highlight the strengths and limitations of each approach and provide a robust foundation for future work on continuous sign recognition and the integration of multimodal learning strategies.