Blind and visually impaired (BVI) learners face challenges accessing image-based educational content. This paper presents an AI-driven system that generates multimodal representations—raised tactile graphics and descriptive text—by integrating open-vocabulary object detection (YOLO-World), image segmentation (SAM), edge detection (Canny) and natural language generation (GPT-4o). The goal is to improve image accessibility for BVI students. A user study with four BVI participants in two educational scenarios (animal anatomy and biology scales) showed that the multimodal outputs enhanced understanding and were not seen as overly complex. Participants appreciated the dual-modality approach (average rating: 4.25/5), though one participant noted a need for practice or assistance. These results, although preliminary, highlight the system’s potential to make educational images more accessible and underscore the need for personalization and user training.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing Image Accessibility in Educational Contexts for Blind and Visually Impaired Learners Through Integrated Computer Vision Techniques

  • Mazen Salous,
  • John-Uwe Riecken,
  • Wilko Heuten,
  • Larbi Abdenaboui

摘要

Blind and visually impaired (BVI) learners face challenges accessing image-based educational content. This paper presents an AI-driven system that generates multimodal representations—raised tactile graphics and descriptive text—by integrating open-vocabulary object detection (YOLO-World), image segmentation (SAM), edge detection (Canny) and natural language generation (GPT-4o). The goal is to improve image accessibility for BVI students. A user study with four BVI participants in two educational scenarios (animal anatomy and biology scales) showed that the multimodal outputs enhanced understanding and were not seen as overly complex. Participants appreciated the dual-modality approach (average rating: 4.25/5), though one participant noted a need for practice or assistance. These results, although preliminary, highlight the system’s potential to make educational images more accessible and underscore the need for personalization and user training.