Human perception relies heavily on vision, which enables people to engage with their environment in meaningful ways. Lack of visual perception makes it extremely difficult for people with visual impairments to navigate and carry out daily duties on their own. For the provision of context-sensitive, sound-related assistance to blind users, in this work, a real-time assistive system is proposed using state-of-the-art language and vision models. Real-time and precise assistance is provided through the system using a modular approach combining efficient item recognition and a language model for a rich scene description. Real-time object detection, visual content semantics, and assistive technology user interface are the primary components delivering an end-to-end assistance system. By addressing the urgent necessity for accessibility, this method provides improved independence and accessibility to the blind group of people.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Integrating Language and Vision Models for Real-Time Assistance for the Visually Impaired

  • Akash Vijayakumar Angadi,
  • Anurag N. Koppal,
  • Arnav Srinivas Atrey,
  • Ch. Rithwik Sai Surya,
  • H. R. Mamatha

摘要

Human perception relies heavily on vision, which enables people to engage with their environment in meaningful ways. Lack of visual perception makes it extremely difficult for people with visual impairments to navigate and carry out daily duties on their own. For the provision of context-sensitive, sound-related assistance to blind users, in this work, a real-time assistive system is proposed using state-of-the-art language and vision models. Real-time and precise assistance is provided through the system using a modular approach combining efficient item recognition and a language model for a rich scene description. Real-time object detection, visual content semantics, and assistive technology user interface are the primary components delivering an end-to-end assistance system. By addressing the urgent necessity for accessibility, this method provides improved independence and accessibility to the blind group of people.