Continuous Indian Sign Language Translator
摘要
This research proposes a deep learning framework for Continuous Indian Sign Language Translation (CISLT), converting sign gestures into natural language text. It fuses spatial and visual features by combining 3D hand landmarks and video frames to capture temporal and semantic patterns in ISL. A custom dataset of eight sentence classes with frame-level word alignments is used for supervised training. Preprocessing involves frame extraction, normalisation, and landmark detection via MediaPipe. The architecture includes ResNet-18 for visual features, an MLP for landmarks, and a Seq2Seq model with spatiotemporal attention for word-level prediction. Data augmentation and precise sequence alignment improve reliability. The model performs well on unseen ISL data, demonstrating its practical and adaptable value for Deaf and hard-of-hearing communication.