Scholarlink

This research introduces a new assistive technology that combines real-time image captioning, voice synthesis, and a model-based keyword-spotting method to help people with visual impairments. In order to identify specified voice instructions as the active trigger and start a camera to record the user’s environment, the system makes use of a lightweight machine learning framework. CLIP, a cutting-edge vision-language model, is used to interpret the visual input and provide contextual textual descriptions of the surroundings. Tacotron 2, a neural text-to-speech algorithm, transforms these captions into natural-sounding speech so that users may hear their environment. The end-to-end pipeline puts usability and low latency first, showing that speech-driven activation, sophisticated picture interpretation, and high-quality audio synthesis can all be combined to provide an easy-to-use assistive tool for practical uses.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Speech-Based Real-World Scene Understanding for Assistive Care of the Visually Impaired

  • 章节

  • 发表日期：

    2025年

查看全文
  • Tarun Sunil,
  • K. Vinod,
  • M. Madhav,
  • Joshua Abraham,
  • G. Jyothish Lal

摘要

This research introduces a new assistive technology that combines real-time image captioning, voice synthesis, and a model-based keyword-spotting method to help people with visual impairments. In order to identify specified voice instructions as the active trigger and start a camera to record the user’s environment, the system makes use of a lightweight machine learning framework. CLIP, a cutting-edge vision-language model, is used to interpret the visual input and provide contextual textual descriptions of the surroundings. Tacotron 2, a neural text-to-speech algorithm, transforms these captions into natural-sounding speech so that users may hear their environment. The end-to-end pipeline puts usability and low latency first, showing that speech-driven activation, sophisticated picture interpretation, and high-quality audio synthesis can all be combined to provide an easy-to-use assistive tool for practical uses.