This paper presents an approach for converting text to Indian Sign Language (ISL) gloss using an encoder-decoder model with an attention mechanism. ISL, like other sign languages, has a distinct grammatical structure that differs significantly from spoken and written languages. The challenge of translating spoken language text into ISL gloss involves addressing complex linguistic features such as word order, tense, negation, and the omission of non-essential words. Our model employs an encoder-decoder framework, where the encoder captures the semantic meaning of input text, and the decoder generates the corresponding ISL gloss sequence. To improve translation accuracy, we incorporate an attention mechanism that dynamically aligns relevant parts of the input text with corresponding gloss tokens, ensuring that contextually important words are prioritized. The work is evaluated using a dataset of 300 ISL gloss sequences, achieving a BLEU score of 0.82, indicating strong translation accuracy. Additionally, the model achieves a Word Error Rate (WER) of 8.5% and a validation accuracy of 95%, demonstrating its robust performance in translating text to ISL gloss. The average conversion time is 0.12 s, highlighting the model’s efficiency for real-time applications. This work contributes to developing effective tools for enhancing communication accessibility within the Deaf and hard-of-hearing communities in India.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Attention-Driven Text-to-Gloss Translation Model for Indian Sign Language

  • Smita Naik,
  • Mrunal Golivadekar,
  • R. Sreemathy,
  • M. P. Turuk

摘要

This paper presents an approach for converting text to Indian Sign Language (ISL) gloss using an encoder-decoder model with an attention mechanism. ISL, like other sign languages, has a distinct grammatical structure that differs significantly from spoken and written languages. The challenge of translating spoken language text into ISL gloss involves addressing complex linguistic features such as word order, tense, negation, and the omission of non-essential words. Our model employs an encoder-decoder framework, where the encoder captures the semantic meaning of input text, and the decoder generates the corresponding ISL gloss sequence. To improve translation accuracy, we incorporate an attention mechanism that dynamically aligns relevant parts of the input text with corresponding gloss tokens, ensuring that contextually important words are prioritized. The work is evaluated using a dataset of 300 ISL gloss sequences, achieving a BLEU score of 0.82, indicating strong translation accuracy. Additionally, the model achieves a Word Error Rate (WER) of 8.5% and a validation accuracy of 95%, demonstrating its robust performance in translating text to ISL gloss. The average conversion time is 0.12 s, highlighting the model’s efficiency for real-time applications. This work contributes to developing effective tools for enhancing communication accessibility within the Deaf and hard-of-hearing communities in India.