Scholarlink

<p>In today’s information-driven world, access to knowledge is essential for personal growth and social inclusion. Yet, individuals with visual impairments, language barriers, or limited literacy often face difficulties accessing printed and digital content. To address these challenges, we propose an inclusive reader system that combines YOLOv8 for document layout analysis, Tesseract for text recognition, Pix2Tex for formula recognition, BLIP2 for image captioning, BLIP2 - after comparison with CLIP - for image-text matching, IndicTrans2 for translation, and gTTS for text-to-speech. The system delivers both textual and visual content through accessible audio formats, supporting multiple languages. It has been tested on English, Tamil, Hindi, and Bengali, with potential for further language expansion. YOLOv8 proved most effective for layout analysis, and BLIP2 preferred over CLIP for compelling reasons. User testing with blind and multilingual participants showed positive feedback, with an average MOS of 4.5+ across languages.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Multimodal book reader with multilingual support

  • 期刊论文

  • 发表日期：

    2026年2月1日

查看全文
  • Rajalakshmi S,
  • Nishaanth R,
  • Shrijith M R,
  • Angeldeborah S

摘要

In today’s information-driven world, access to knowledge is essential for personal growth and social inclusion. Yet, individuals with visual impairments, language barriers, or limited literacy often face difficulties accessing printed and digital content. To address these challenges, we propose an inclusive reader system that combines YOLOv8 for document layout analysis, Tesseract for text recognition, Pix2Tex for formula recognition, BLIP2 for image captioning, BLIP2 - after comparison with CLIP - for image-text matching, IndicTrans2 for translation, and gTTS for text-to-speech. The system delivers both textual and visual content through accessible audio formats, supporting multiple languages. It has been tested on English, Tamil, Hindi, and Bengali, with potential for further language expansion. YOLOv8 proved most effective for layout analysis, and BLIP2 preferred over CLIP for compelling reasons. User testing with blind and multilingual participants showed positive feedback, with an average MOS of 4.5+ across languages.