Hand Gesture Classification on Custom Abductees-Rescue Dataset Using an Optimized LSTM
摘要
This paper presents a solution for classifying abduction-related hand gestures in surveillance systems through the development of a specialized dataset and an optimized LSTM model. Previous research on hand gesture classification for security applications, particularly those focused on distress signals and abduction detection, has utilized datasets with significant practical limitations. These limitations include data captured via mobile phone or laptop front-facing cameras rather than surveillance equipment, restricted detection ranges, and predominantly controlled lighting conditions that inadequately represent real-world surveillance environments. To address these limitations, the Abductees-Rescue dataset has been developed using surveillance cameras mounted at 3-m heights. Following preprocessing and landmark extraction, the dataset has yielded 9,111 samples (4,545 normal hand gestures and 4,566 abduction signals). Each sample contains sequential data from 45 frames of hand videos, where each frame provides 21 three-dimensional landmarks extracted by using MediaPipe Hand framework. An effective preprocessing approach normalizes these landmarks relative to the wrist joint, eliminating variations in hand position and size. The classification model implements a hierarchical Long Short-Term Memory (LSTM) architecture with L2 regularization, batch normalization, and optimized dropout layers. Experimental results demonstrate 95.06% accuracy, 95.24% precision, and 94.89% recall on the test dataset, with balanced performance across both gesture categories. The confusion matrix analysis shows a proportional distribution of misclassifications, indicating robust generalization capability. This research contributes to the field of security surveillance by providing both a standardized dataset and an effective classification methodology for detecting potential abduction situations through hand gesture analysis.