RAG in Research: Evaluating AI-Driven Literature Search Tools
摘要
Retrieval-augmented generation (RAG) tools aim to combine language models and real-time database search to mitigate AI hallucinations. This study, conducted in the fall of 2024, assessed four academic RAG tools with respect to their ease of use as well as relevance of search results. The evaluation included a survey and a series of workshops with researchers at KTH Royal Institute of Technology, as well as additional tests conducted by library staff following a structured search protocol. Our findings suggest that, at this stage, these tools cannot replace traditional systematic database searches. A key weakness of these tools is the lack of context provided with the search results. We argue, in conclusion, for establishing some guiding principles for appropriate use of RAG tools in different use cases.