The integration of large language models (LLMs) with external sources has established Retrieval-Augmented Generation (RAG) as a key technology for improving reliability and reducing hallucinations. While RAG has proven effective, the paradigm has evolved to incorporate various techniques at different stages, aiming to further enhance its potential. However, a key challenge lies in identifying configurations of advanced techniques that improve performance. This study evaluates several technique configurations in a modular RAG system, with the objective of identifying those that deliver the best performance. Technologies such as LangChain, ChromaDB, and LLaMA 3.1:8B are employed. The evaluation focuses on metrics such as precision, recall, and F1-Score using a dataset from the academic postgraduate domain. The results highlight variability across configurations and emphasize how properly designing the processing of retrieved context can improve the quality of generated answers.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Generative AI-Based Virtual Assistant Using Retrieval-Augmented Generation: An Evaluation Study

  • Claudia Farrada Machado,
  • Alfredo Simón-Cuevas,
  • Neili Machado García

摘要

The integration of large language models (LLMs) with external sources has established Retrieval-Augmented Generation (RAG) as a key technology for improving reliability and reducing hallucinations. While RAG has proven effective, the paradigm has evolved to incorporate various techniques at different stages, aiming to further enhance its potential. However, a key challenge lies in identifying configurations of advanced techniques that improve performance. This study evaluates several technique configurations in a modular RAG system, with the objective of identifying those that deliver the best performance. Technologies such as LangChain, ChromaDB, and LLaMA 3.1:8B are employed. The evaluation focuses on metrics such as precision, recall, and F1-Score using a dataset from the academic postgraduate domain. The results highlight variability across configurations and emphasize how properly designing the processing of retrieved context can improve the quality of generated answers.