Scholarlink

<p>Despite of swift advancements in the field of Artificial Intelligence, an effective immediate assistive solution for visually impaired individuals remains limited. Whereas, Deep Learning and Natural Language Processing (NLP) have attained significant gain in visual sympathetic and language generation but their addition into accessible systems for ecological perception is still underexplored. This dictates the development of intelligent frameworks capable of accurately rendering visual scenes and delivering meaningful descriptions to enhance liberation and situational awareness for visually impaired users. Consequently, we proposed a framework of three levels using the deep learning and NLP approaches to address aforementioned. It also developed a novel approach to learning the better relational features among the objects, scenes, and persons captured in an image and generated an accurate caption. Firstly, object detection algorithms are used to detect the objects in an image. The next level generated the image captions by maximizing the likelihood of the expected captions using deep learning. The third level, the text in the caption, is converted into a voice using NLP algorithms. The proposed model has substantially compared the existing state-of-the-art image captioning and voice conversion methods by experimenting with benchmark datasets such as Flickr 8k, Flickr 30k, COCO Caption, and RyanSpeech. The proposed model outperformed in terms of recall on Flickr 8K dataset @100 images has scored 67.25% and 69.10% rated on the same dataset @200 images. In addition, BLEU − 1, 2 3, and 4 are scored 71.25%, 69.2%, 54.7%, and 40.8%.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

An assistive framework for visually impaired using deep learning and natural language processing

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月14日

查看全文
  • V. Uma Maheswari,
  • Rajanikanth Aluvalu,
  • Rajesh Kumar Dhanaraj,
  • Mahmoud Ahmad Al-Khasawneh,
  • Hariprasath Manoharan,
  • Shitharth Selvarajan

摘要

Despite of swift advancements in the field of Artificial Intelligence, an effective immediate assistive solution for visually impaired individuals remains limited. Whereas, Deep Learning and Natural Language Processing (NLP) have attained significant gain in visual sympathetic and language generation but their addition into accessible systems for ecological perception is still underexplored. This dictates the development of intelligent frameworks capable of accurately rendering visual scenes and delivering meaningful descriptions to enhance liberation and situational awareness for visually impaired users. Consequently, we proposed a framework of three levels using the deep learning and NLP approaches to address aforementioned. It also developed a novel approach to learning the better relational features among the objects, scenes, and persons captured in an image and generated an accurate caption. Firstly, object detection algorithms are used to detect the objects in an image. The next level generated the image captions by maximizing the likelihood of the expected captions using deep learning. The third level, the text in the caption, is converted into a voice using NLP algorithms. The proposed model has substantially compared the existing state-of-the-art image captioning and voice conversion methods by experimenting with benchmark datasets such as Flickr 8k, Flickr 30k, COCO Caption, and RyanSpeech. The proposed model outperformed in terms of recall on Flickr 8K dataset @100 images has scored 67.25% and 69.10% rated on the same dataset @200 images. In addition, BLEU − 1, 2 3, and 4 are scored 71.25%, 69.2%, 54.7%, and 40.8%.