Scholarlink

<p>Recent advancements in large video models (LVMs) have significantly enhanced video understanding. However, these models continue to suffer from hallucinations, producing content that conflicts with input videos. To address this issue, we propose <b>Dr.V</b>, a hierarchical framework covering perceptive, temporal, and cognitive levels to diagnose and mitigate video hallucination by fine-grained spatial-temporal grounding. Dr.V comprises two key components: a benchmark dataset <b>Dr.V-Bench</b>, and a satellite video agent <b>Dr.V-Agent</b>. Dr.V-Bench includes 10k instances drawn from 4,974 videos spanning diverse tasks, each enriched with detailed spatial-temporal annotation. Dr.V-Agent detects hallucinations in LVMs by systematically applying fine-grained spatial-temporal grounding at the perceptive and temporal levels, followed by cognitive level reasoning, and subsequently guides the LVMs to correct their responses through structured feedback. This step-by-step pipeline mirrors human-like video comprehension and effectively identifies and rectifies hallucinations. Extensive experiments demonstrate that Dr.V-Agent is effective in both diagnosing and mitigating hallucination while enhancing interpretability and reliability, offering a practical blueprint for robust video understanding in real-world scenarios. All our data and code (<a href="https://github.com/Eurekaleo/Dr.V">https://github.com/Eurekaleo/Dr.V</a>) will be made open to facilitate future research.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Dr.V : A Hierarchical Perception-Temporal-Cognition Framework to Diagnose Video Hallucination by Fine-Grained Spatial-Temporal Grounding

  • 期刊论文

  • 发表日期：

    2026年5月20日

查看全文
  • Meng Luo,
  • Shengqiong Wu,
  • Liqiang Jing,
  • Tianjie Ju,
  • Li Zheng,
  • Jinxiang Lai,
  • Tianlong Wu,
  • Xinya Du,
  • Jian Li,
  • Siyuan Yan,
  • Jiebo Luo,
  • William Yang Wang,
  • Hao Fei,
  • Mong-Li Lee,
  • Wynne Hsu

摘要

Recent advancements in large video models (LVMs) have significantly enhanced video understanding. However, these models continue to suffer from hallucinations, producing content that conflicts with input videos. To address this issue, we propose Dr.V, a hierarchical framework covering perceptive, temporal, and cognitive levels to diagnose and mitigate video hallucination by fine-grained spatial-temporal grounding. Dr.V comprises two key components: a benchmark dataset Dr.V-Bench, and a satellite video agent Dr.V-Agent. Dr.V-Bench includes 10k instances drawn from 4,974 videos spanning diverse tasks, each enriched with detailed spatial-temporal annotation. Dr.V-Agent detects hallucinations in LVMs by systematically applying fine-grained spatial-temporal grounding at the perceptive and temporal levels, followed by cognitive level reasoning, and subsequently guides the LVMs to correct their responses through structured feedback. This step-by-step pipeline mirrors human-like video comprehension and effectively identifies and rectifies hallucinations. Extensive experiments demonstrate that Dr.V-Agent is effective in both diagnosing and mitigating hallucination while enhancing interpretability and reliability, offering a practical blueprint for robust video understanding in real-world scenarios. All our data and code (https://github.com/Eurekaleo/Dr.V) will be made open to facilitate future research.