Scholarlink

<p>With the rapid advancement of multimodal foundation models, artificial intelligence (AI) has shown growing potential in computational pathology. However, current approaches still rely heavily on large annotated datasets, while real-world pathological data are limited and generalize poorly across cohorts. To address these issues, we systematically evaluate state-of-the-art large multimodal models (LMMs) on pathology-related tasks under zero-shot and few-shot settings. We further propose a retrieval-augmented support-set prompting framework that incorporates class-aware retrieved image-text exemplars to enhance contextual reasoning and pathological image understanding. By using class-aware retrieved image-text exemplars as external evidence, the framework enables models to capture fine-grained pathological patterns and produce more consistent and interpretable predictions. Extensive experiments on multiple pathology datasets demonstrate significant performance gains, highlighting the effectiveness of retrieval-based contextual augmentation in improving both reasoning ability in complex medical domains. The code and datasets are available at <a href="https://github.com/ttchu1221/RALM-Path">https://github.com/ttchu1221/RALM-Path</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Retrieval-augmented large multimodal models for pathology

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年6月23日

查看全文
  • Xinhui Chu,
  • Yuxuan Sun,
  • Binbin Zhou,
  • Lin Sun

摘要

With the rapid advancement of multimodal foundation models, artificial intelligence (AI) has shown growing potential in computational pathology. However, current approaches still rely heavily on large annotated datasets, while real-world pathological data are limited and generalize poorly across cohorts. To address these issues, we systematically evaluate state-of-the-art large multimodal models (LMMs) on pathology-related tasks under zero-shot and few-shot settings. We further propose a retrieval-augmented support-set prompting framework that incorporates class-aware retrieved image-text exemplars to enhance contextual reasoning and pathological image understanding. By using class-aware retrieved image-text exemplars as external evidence, the framework enables models to capture fine-grained pathological patterns and produce more consistent and interpretable predictions. Extensive experiments on multiple pathology datasets demonstrate significant performance gains, highlighting the effectiveness of retrieval-based contextual augmentation in improving both reasoning ability in complex medical domains. The code and datasets are available at https://github.com/ttchu1221/RALM-Path.