Recent years have witnessed a rapid development of Large Vision-Language Models (LVLMs), spurred by progress in Large Language Models (LLMs). However, a significant challenge has emerged: Object Hallucination, where the generated textual descriptions misalign with the factual visual content. This issue poses a critical safety risk in applications requiring high accuracy and hampers further progress in LVLM development. While previous research has partially addressed hallucinations from a visual standpoint, the language model capability within LVLMs remains underutilized. In this work, we introduce Assembled Chain-of-Thought (A-CoT), a novel training-free method designed to mitigate object hallucinations through effective Chain-of-Thought (CoT) reasoning. By using diverse Chain-of-Thought prompts to leverage knowledge in LVLMs and integrating them through assembly decoding, A-CoT combines original and CoT outputs, enabling multi-perspective reasoning, enhancing output robustness, and significantly advancing hallucination mitigation techniques. Experiments on multiple object hallucination benchmarks—POPE, CHAIR, and MME Hallucination Subset—demonstrate that A-CoT significantly reduces object hallucinations and outperforms baseline models. The code is available at https://github.com/pianopiece/ACoT .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Mitigating Object Hallucination Through Assembled Chain-of-Thought Reasoning

  • Xinhao Wang,
  • Xinyu Ma,
  • Shengyong Ding,
  • Lidia S. Chao,
  • Derek F. Wong

摘要

Recent years have witnessed a rapid development of Large Vision-Language Models (LVLMs), spurred by progress in Large Language Models (LLMs). However, a significant challenge has emerged: Object Hallucination, where the generated textual descriptions misalign with the factual visual content. This issue poses a critical safety risk in applications requiring high accuracy and hampers further progress in LVLM development. While previous research has partially addressed hallucinations from a visual standpoint, the language model capability within LVLMs remains underutilized. In this work, we introduce Assembled Chain-of-Thought (A-CoT), a novel training-free method designed to mitigate object hallucinations through effective Chain-of-Thought (CoT) reasoning. By using diverse Chain-of-Thought prompts to leverage knowledge in LVLMs and integrating them through assembly decoding, A-CoT combines original and CoT outputs, enabling multi-perspective reasoning, enhancing output robustness, and significantly advancing hallucination mitigation techniques. Experiments on multiple object hallucination benchmarks—POPE, CHAIR, and MME Hallucination Subset—demonstrate that A-CoT significantly reduces object hallucinations and outperforms baseline models. The code is available at https://github.com/pianopiece/ACoT .