Scholarlink

This study introduces a RAG-based intelligent tutoring system enhanced to process multimodal data for higher education, addressing the limitations of traditional text-based LLM tutoring tools. While the previous version of the system focused on text responses, feedback from other International Conferences has highlighted the need for integrating additional data types like images and videos to foster a richer and more interactive learning experience. The proposed system combines LangChain, Large Vision Language Models (LVLMs), and the BridgeTower embedding model, creating unified representations of text, image, and video content. Educational resources are stored in high-dimensional vector stores, enabling efficient retrieval and contextually relevant responses to students’ diverse queries. By expanding the Intelligent Agent’s capabilities to interpret visual content, this system aims to improve engagement, comprehension, and retention of complex academic material. Early testing has shown encouraging results in text processing, and similar success is anticipated with multimodal integration. This research demonstrates the potential for RAG-based multimodal tutoring to significantly enhance learning in higher education and recommends continued development of these capabilities for broader educational applications.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Enhancing Higher Education with Multimodal Intelligent Agent Using a RAG-Based Approach

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Horia Alexandru Modran,
  • Ioana Corina Bogdan,
  • Paul Livius Modran

摘要

This study introduces a RAG-based intelligent tutoring system enhanced to process multimodal data for higher education, addressing the limitations of traditional text-based LLM tutoring tools. While the previous version of the system focused on text responses, feedback from other International Conferences has highlighted the need for integrating additional data types like images and videos to foster a richer and more interactive learning experience. The proposed system combines LangChain, Large Vision Language Models (LVLMs), and the BridgeTower embedding model, creating unified representations of text, image, and video content. Educational resources are stored in high-dimensional vector stores, enabling efficient retrieval and contextually relevant responses to students’ diverse queries. By expanding the Intelligent Agent’s capabilities to interpret visual content, this system aims to improve engagement, comprehension, and retention of complex academic material. Early testing has shown encouraging results in text processing, and similar success is anticipated with multimodal integration. This research demonstrates the potential for RAG-based multimodal tutoring to significantly enhance learning in higher education and recommends continued development of these capabilities for broader educational applications.