Scholarlink

<p>Understanding scientific figures is challenging due to their complexity and the need for domain expertise. Existing studies, limited by data and model capabilities, primarily focus on captioning and classifying simpler visualizations, such as bar and pie charts, while neglecting the fine-grained interpretation of knowledge-rich diagrams like frameworks and flowcharts. To address this gap, we introduce a novel task, <i>Figure Integrity Verification</i>, to assess models’ ability to align textual knowledge with detailed visual elements and identify visual elements that lack textual explanations. To support this task, we construct a large-scale text-figure alignment dataset, <i>Figure-seg</i>, containing 15,761 instances. Building upon this dataset, we propose an innovative framework, Every Part Matters (EPM), which employs Multimodal Large Language Models (MLLMs) and analogical reasoning to verify and enhance figure integrity. Experimental results demonstrate that the proposed framework improves the IoU metric for text-figure alignment by 33.83% and for figure integrity verification by 4.71% compared to state-of-the-art (SOTA) methods. This progress establishes a new benchmark for scientific figure understanding and advances practical applications in fields requiring the accurate interpretation of complex visual data in scientific research (All data is available open-source and instructions for downloading the data are available at <a href="https://github.com/shixiang1a/figure_understanding">https://github.com/shixiang1a/figure_understanding</a>).</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Figure-seg: a new benchmark for understanding scientific figures via integrity verification

  • 期刊论文

  • 发表日期：

    2026年5月27日

查看全文
  • Xiang Shi,
  • Jiawei Liu,
  • Yinpeng Liu,
  • Zhu Liang,
  • Qikai Cheng,
  • Wei Lu,
  • Rui Zhang

摘要

Understanding scientific figures is challenging due to their complexity and the need for domain expertise. Existing studies, limited by data and model capabilities, primarily focus on captioning and classifying simpler visualizations, such as bar and pie charts, while neglecting the fine-grained interpretation of knowledge-rich diagrams like frameworks and flowcharts. To address this gap, we introduce a novel task, Figure Integrity Verification, to assess models’ ability to align textual knowledge with detailed visual elements and identify visual elements that lack textual explanations. To support this task, we construct a large-scale text-figure alignment dataset, Figure-seg, containing 15,761 instances. Building upon this dataset, we propose an innovative framework, Every Part Matters (EPM), which employs Multimodal Large Language Models (MLLMs) and analogical reasoning to verify and enhance figure integrity. Experimental results demonstrate that the proposed framework improves the IoU metric for text-figure alignment by 33.83% and for figure integrity verification by 4.71% compared to state-of-the-art (SOTA) methods. This progress establishes a new benchmark for scientific figure understanding and advances practical applications in fields requiring the accurate interpretation of complex visual data in scientific research (All data is available open-source and instructions for downloading the data are available at https://github.com/shixiang1a/figure_understanding).