Retrieval-augmented generation (RAG) is a framework designed to address the limitation of large language models (LLM) in terms of business domain awareness and knowledge cutoff. Hence, the adoption of RAG has been immense in recent times as it can overcome the above challenges. However, RAG also consists of several techniques and challenges. A few common challenges include being unable to produce optimal response or output format mismatch, missing to refer most important sources, and incapable of retrieving the appropriate paragraphs or contexts. As a result, the response accuracy of RAG-based applications deteriorates. Hence, a recommendation system is the need of the hour which can assist the users to choose the most appropriate method based on the specific scenario. In this paper, an end-to-end RAG-based application performance improvement framework is proposed which will assist the users to select the optimal approach based on the present evaluation score and other constraints. Evaluation score is calculated based on the well-known metrices which include groundedness, answer relevance, and context relevance. The framework is a collection of different techniques applied iteratively at different stages of RAG with the goal of improving the overall score. The embedding being the backbone of the RAG, a right fit embedding model recommendation is part of the overall framework.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

An End-to-End Framework Toward Improving Retrieval-Augmented Generation (RAG)-Based Application Performance

  • Aritra Sen,
  • Anindita Desarkar,
  • Vishwanathan Raman

摘要

Retrieval-augmented generation (RAG) is a framework designed to address the limitation of large language models (LLM) in terms of business domain awareness and knowledge cutoff. Hence, the adoption of RAG has been immense in recent times as it can overcome the above challenges. However, RAG also consists of several techniques and challenges. A few common challenges include being unable to produce optimal response or output format mismatch, missing to refer most important sources, and incapable of retrieving the appropriate paragraphs or contexts. As a result, the response accuracy of RAG-based applications deteriorates. Hence, a recommendation system is the need of the hour which can assist the users to choose the most appropriate method based on the specific scenario. In this paper, an end-to-end RAG-based application performance improvement framework is proposed which will assist the users to select the optimal approach based on the present evaluation score and other constraints. Evaluation score is calculated based on the well-known metrices which include groundedness, answer relevance, and context relevance. The framework is a collection of different techniques applied iteratively at different stages of RAG with the goal of improving the overall score. The embedding being the backbone of the RAG, a right fit embedding model recommendation is part of the overall framework.