Abstract <p>An innovative approach for generating Bengali captions from images is introduced in this study, to enhance regional language understanding and vision-language comprehension. Leveraging a substantial Bengali dataset, optimization techniques are incorporated to ensure the efficient implementation of the model. Image features are extracted using established models, including ResNet-50, while captions undergo preprocessing through natural language processing (NLP) techniques. Improved performance is demonstrated by the proposed scheme, which integrates both image and text features, with average parameter values, including BLUE 1: 0.736477, BLUE 2: 0.470831, BLUE 3: 0.252285, BLUE 4: 0.081330, ROUGE: 0.469742, and METEOR: 0.245380. A significant contribution to bridging the gap between regional languages and AI-driven technologies is made by this research, fostering advancements in Bengali vision-language comprehension. The model is designed and experimented with the selection of either InceptionV3, VGG16, or ResNet-50. This work specifically addresses the lack of regionally adapted image captioning systems for Bengali by benchmarking multiple CNN backbones and refining NLP pipelines to tailor performance for a low-resource setting.</p> Graphic abstract <p></p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Advancing Bengali Vision-Language Comprehension: A Deep Learning and NLP Paradigm for Image Captioning

  • Dipankar Dey,
  • Dipak Kumar Jana,
  • Prajna Bhunia

摘要

Abstract

An innovative approach for generating Bengali captions from images is introduced in this study, to enhance regional language understanding and vision-language comprehension. Leveraging a substantial Bengali dataset, optimization techniques are incorporated to ensure the efficient implementation of the model. Image features are extracted using established models, including ResNet-50, while captions undergo preprocessing through natural language processing (NLP) techniques. Improved performance is demonstrated by the proposed scheme, which integrates both image and text features, with average parameter values, including BLUE 1: 0.736477, BLUE 2: 0.470831, BLUE 3: 0.252285, BLUE 4: 0.081330, ROUGE: 0.469742, and METEOR: 0.245380. A significant contribution to bridging the gap between regional languages and AI-driven technologies is made by this research, fostering advancements in Bengali vision-language comprehension. The model is designed and experimented with the selection of either InceptionV3, VGG16, or ResNet-50. This work specifically addresses the lack of regionally adapted image captioning systems for Bengali by benchmarking multiple CNN backbones and refining NLP pipelines to tailor performance for a low-resource setting.

Graphic abstract