Large Language Models (LLMs) excel at open-domain question answering but struggle with domain-specific queries, particularly involving dynamic and specialized financial data. This paper introduces a Multi-Agent Retrieval-Augmented Generation (RAG) system specifically designed for processing complex Arabic financial reports from the Central Bank of Libya. Recognizing the challenges posed by these documents, our approach incorporates rigorous preprocessing to create a high-fidelity dataset. The core system employs a hierarchical framework where a top-level agent directs specialized low-level agents for refined, domain-targeted searches, enhanced by an iterative refinement mechanism for handling complex multi-hop queries. Comparative evaluations against a Naive RAG baseline using metrics including Answer Correctness, Faithfulness, and Relevance demonstrate our system’s effectiveness. Notably, the Multi–Agent RAG system achieved an answer correctness score of (3.72/5), significantly outperforming the Naive RAG baseline (2.31/5). This highlights the advantages of a structured, multi-agent retrieval process for achieving higher accuracy and reliability when querying complex financial information.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Developing and Evaluating Multi-agent RAG System for Central Bank of Libya Financial Reports

  • Abdulrauf Zoubi,
  • Maram Manita,
  • Rayhan Alshwehdi,
  • Esmaeil Alkhazmi,
  • Miftah Najeeb,
  • Salma Elkawafi

摘要

Large Language Models (LLMs) excel at open-domain question answering but struggle with domain-specific queries, particularly involving dynamic and specialized financial data. This paper introduces a Multi-Agent Retrieval-Augmented Generation (RAG) system specifically designed for processing complex Arabic financial reports from the Central Bank of Libya. Recognizing the challenges posed by these documents, our approach incorporates rigorous preprocessing to create a high-fidelity dataset. The core system employs a hierarchical framework where a top-level agent directs specialized low-level agents for refined, domain-targeted searches, enhanced by an iterative refinement mechanism for handling complex multi-hop queries. Comparative evaluations against a Naive RAG baseline using metrics including Answer Correctness, Faithfulness, and Relevance demonstrate our system’s effectiveness. Notably, the Multi–Agent RAG system achieved an answer correctness score of (3.72/5), significantly outperforming the Naive RAG baseline (2.31/5). This highlights the advantages of a structured, multi-agent retrieval process for achieving higher accuracy and reliability when querying complex financial information.