Olympic Chatbot Using ReAct Agents
摘要
The Olympic Games attract a large audience all over the world with diverse sports interests. However, there is a lack of interactive platforms, such as chatbots, for accurate Olympic information. This paper proposes the development of a chatbot specifically for Olympic enthusiasts, focusing on Athletics, Aquatics, and Shooting. The Large Language Modules (LLM) powered chatbot is designed to answer user queries regarding Olympics including schedules, venues, historical records, athlete profiles, and latest updates. We introduce a Corrective Retrieval-Augmented Generation (CRAG) pipeline combined with a Self-RAG mechanism, incorporating dynamic retrieval, grading, and web augmentation via Tavily search when domain coverage is insufficient. A Chroma-based vector store populated with embeddings generated from “BAAI/bge-m3” (1024-dimension) models ensures high-fidelity semantic retrieval. The workflow consists of creating vector stores, document retrieval of the user query by the use of vectorization, grading documents on hallucinations and correct retrieval which is powered by Mistral AI, web search with the help of agents, and checking the groundedness of the final answer generated with the user query. We benchmarked Naive RAG against our Agentic RAG framework, achieving substantial improvements across four evaluation metrics: Response Relevancy (82.55%), Context Recall (86.00%), Context Precision (87.43%), and Faithfulness (81.07%). An Olympedia-derived dataset comprising 46,000 athlete profiles and 2,000 event records was used to create a 150,000-question evaluation set, ensuring robust validation.