Graph Summarization for Educational RAG: Two-Stage Retrieval with Structured and Multimodal Inputs
摘要
Retrieval-Augmented Generation (RAG) has proven effective in educational contexts for improving the factual grounding of large language models (LLMs), yet most systems rely solely on unstructured text retrieval. While recent approaches introduce knowledge graphs for better structure, they often fail to address complex, multimodal queries essential to effective instruction. In response, we propose a graph-enhanced RAG system that integrates text and image-derived instructional materials—such as lecture slides and diagrams—into a unified knowledge graph. The system employs a dual-stage pipeline: first retrieving content via semantic and graph-based search, then summarizing it into prompt-efficient inputs for generation. This design supports pedagogically aligned, explainable answers and enhances transparency through source-level similarity tracing. We validate our system using a dataset derived from lecture slides in a Master’s-level deep learning course, comparing three configurations: a baseline text-only retriever, a dual-stage summarizer, and a full multimodal variant with dual-stage summarization using both text and image inputs. Results show that combining multimodal retrieval with graph-based summarization produces more coherent and context-rich responses, particularly for visually grounded or conceptually layered questions.