Scholarlink

This research paper develops a multimodal translation system that is versatile in nature and specifically designed for converting English-to-Telugu language, which addresses accessibility issues across various applications. Our method involves using ResNet50 to ensure that we have robust image feature extraction while using word tokenization to generate English and Telugu tokens. The translation is done through FNet encoder-decoder architectures with the multimodal encoder acting as the de facto combiner of textual and visual information and the decoder generating the Telugu translation to ensure the results are both accurate and contextually appropriate. Based on extensive experimental observations, our model achieved a BLEU score of 54.73, significantly outperforming unimodal approaches such as LSTM (28.97), GRU (35.61), and transformer (46.22) models. Our framework goes beyond assistive devices for the blind and has the possibility of being used as a navigation aid, communication tool, or educational resource thereby improving access and inclusion among individuals who speak different languages.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A Multimodal Approach for English-to-Telugu Translation with FNet and Transformer Models

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Joshita Malla,
  • Ujwala Kanumuri,
  • Swathi Mytreye Chaganti,
  • J. Swaminathan

摘要

This research paper develops a multimodal translation system that is versatile in nature and specifically designed for converting English-to-Telugu language, which addresses accessibility issues across various applications. Our method involves using ResNet50 to ensure that we have robust image feature extraction while using word tokenization to generate English and Telugu tokens. The translation is done through FNet encoder-decoder architectures with the multimodal encoder acting as the de facto combiner of textual and visual information and the decoder generating the Telugu translation to ensure the results are both accurate and contextually appropriate. Based on extensive experimental observations, our model achieved a BLEU score of 54.73, significantly outperforming unimodal approaches such as LSTM (28.97), GRU (35.61), and transformer (46.22) models. Our framework goes beyond assistive devices for the blind and has the possibility of being used as a navigation aid, communication tool, or educational resource thereby improving access and inclusion among individuals who speak different languages.