Cross Model Communication Sign Language to Text and Speech to Sign Language Using Inception V5
摘要
Cross model communication, a bidirectional sign language translation system that bridges communication between American Sign Language (ASL) users and non-signers. The system integrates two core modules: (1) a sign-to-text translation pipeline using an enhanced Inception V5 model, and (2) a speech-to-sign translation mechanism utilizing Whisper AI for speech recognition and animated ASL output generation. The Inception V5 model classifies ASL gestures with 90.2% accuracy and an inference time of 50 ms, while the speech-to-ASL module transcribes spoken input and maps it to ASL representations with 94% accuracy. A key contribution of this work lies in its modular architecture, which supports dynamic gesture recognition, robust speech transcription under noisy conditions, and multimodal translation output, including static letter signs and animated word signs. Extensive experimentation confirms the system’s reliability across varied lighting conditions, hand orientations, and speech accents. This solution has potential for scalable deployment in educational, social, and assistive contexts. Future enhancements include expanding vocabulary, real-time animated synthesis, and contextual gesture interpretation.