Answer Generation for Large-Scale Official Trial Tests in Japanese University Entrance Exams Using RAG
摘要
We propose generating answers from the Japanese language section for large-scale official trial tests in Japanese university entrance examinations using Retrieval-Augmented Generation (RAG). RAG follows a process in which text segments with high similarity to a given query are retrieved from a database and incorporated into the prompt. Using the question as the query and appending similar segments from the main text to the prompt, we expect to improve the answer generation performance. To investigate this, we used large-scale official trial tests from Japanese university entrance examinations as a benchmark and generated answers using several large language models (LLMs) with and without RAG. The generated answers are then evaluated on the basis of the official scoring conditions. Experiments conducted with gemma2, llaMA3, and gpt-4o show that RAG is effective for relatively small and medium sized LLMs. However, for large-scale LLMs, using the model alone without RAG results in better answer generation. We further propose an enhanced version of RAG, named Re-retrieval RAG, which incorporates a recursive retrieval process to improve the accuracy and consistency of the generated answers. Our experimental results show that the proposed Re-retrieval RAG consistently achieved the highest scores across all six questions tested, demonstrating its general effectiveness.