Sign language generation and recognition are essential for improving accessibility for the deaf and hard-of-hearing community. This study explores deep learning techniques, combining skeleton-based models and StackGAN to synthesize high-quality Indian Sign Language (ISL) representations. Skeletal structures are first extracted to capture hand and finger movements, though they lack fine details like finger shapes and facial expressions. To address this, a two-stage StackGAN refines low-resolution skeletons into realistic ISL signs. The model, evaluated using qualitative and quantitative metrics, effectively synthesizes ISL alphabets and numbers, with generator and discriminator losses of 47% and 51%, respectively. While the results closely resemble real ISL gestures, minor distortions remain due to limited data. Future work will expand the model to word-level sequences, dynamic gestures, and multimodal learning for real-time applications, laying the groundwork for robust ISL communication systems.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Gesture Synthesis for Sign Language Using Generative Adversarial Networks

  • Sakshi Narayan Habib,
  • C. Sujatha,
  • Padmashree Desai

摘要

Sign language generation and recognition are essential for improving accessibility for the deaf and hard-of-hearing community. This study explores deep learning techniques, combining skeleton-based models and StackGAN to synthesize high-quality Indian Sign Language (ISL) representations. Skeletal structures are first extracted to capture hand and finger movements, though they lack fine details like finger shapes and facial expressions. To address this, a two-stage StackGAN refines low-resolution skeletons into realistic ISL signs. The model, evaluated using qualitative and quantitative metrics, effectively synthesizes ISL alphabets and numbers, with generator and discriminator losses of 47% and 51%, respectively. While the results closely resemble real ISL gestures, minor distortions remain due to limited data. Future work will expand the model to word-level sequences, dynamic gestures, and multimodal learning for real-time applications, laying the groundwork for robust ISL communication systems.