Scholarlink

<p>As vision-language models scale, multimodal information retrieval increasingly encounters the bottleneck of complex, multi-hop reasoning that demands domain-specific knowledge integration. Existing Retrieval-Augmented Generation (RAG) systems predominantly rely on static “Retrieve-then-Read” pipelines, which exhibit severe cognitive inertia, brittle visual grounding, and susceptibility to catastrophic forgetting during domain adaptation. To overcome these limitations, we introduce the Agent and Incremental Learning-based Multimodal Information Retrieval (AILMIR) framework, shifting the paradigm toward a dynamic Plan-Execute-Reflect cognitive loop. Operating within a lightweight parameter regime, AILMIR deploys a Planning Agent for hierarchical task decomposition and dynamic contextual prompting. Crucially, a metacognitive Reflection module systematically evaluates the logical consistency and factual faithfulness of retrieved evidence to mitigate multimodal hallucinations. Furthermore, we propose a Non-Parametric Case-Based Memory that sediments successful reasoning trajectories, enabling efficient Domain Incremental Learning (DIL) without destructive gradient updates. Extensive empirical evaluations on the expert-level MMMU and ScienceQA benchmarks validate the superiority of our framework, demonstrating significant improvements in reasoning accuracy and factual grounding alongside near-zero catastrophic forgetting across sequential domain shifts. The source code of AILMIR is available at <a href="https://github.com/SWU-CS-MediaLab/AILMIR">https://github.com/SWU-CS-MediaLab/AILMIR</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

AILMIR: Agent and incremental learning-based multimodal information retrieval for complex multi-hop multimedia queries

  • 期刊论文

  • 发表日期：

    2026年8月3日

查看全文
  • Yichen Fan,
  • Zihan Yang,
  • Haitao Qin,
  • Song Wu

摘要

As vision-language models scale, multimodal information retrieval increasingly encounters the bottleneck of complex, multi-hop reasoning that demands domain-specific knowledge integration. Existing Retrieval-Augmented Generation (RAG) systems predominantly rely on static “Retrieve-then-Read” pipelines, which exhibit severe cognitive inertia, brittle visual grounding, and susceptibility to catastrophic forgetting during domain adaptation. To overcome these limitations, we introduce the Agent and Incremental Learning-based Multimodal Information Retrieval (AILMIR) framework, shifting the paradigm toward a dynamic Plan-Execute-Reflect cognitive loop. Operating within a lightweight parameter regime, AILMIR deploys a Planning Agent for hierarchical task decomposition and dynamic contextual prompting. Crucially, a metacognitive Reflection module systematically evaluates the logical consistency and factual faithfulness of retrieved evidence to mitigate multimodal hallucinations. Furthermore, we propose a Non-Parametric Case-Based Memory that sediments successful reasoning trajectories, enabling efficient Domain Incremental Learning (DIL) without destructive gradient updates. Extensive empirical evaluations on the expert-level MMMU and ScienceQA benchmarks validate the superiority of our framework, demonstrating significant improvements in reasoning accuracy and factual grounding alongside near-zero catastrophic forgetting across sequential domain shifts. The source code of AILMIR is available at https://github.com/SWU-CS-MediaLab/AILMIR.