VDIS: Combating Object Hallucination in Multimodal Large Language Models
摘要
Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities across various multimodal tasks, promising a bright future for artificial intelligence. However, their widespread use is hindered by the persistent issue of object hallucination, where models generate responses misaligned with visual input. To address this, we propose Visual Continuity Disruption (VDIS), a training-free, plug-and-play contrastive decoding method. This approach is inspired by our observation that disrupting visual continuity destabilizes visual tokens, forcing the model to rely more heavily on language priors and statistical biases—key contributors to object hallucinations—during inference. VDIS works by disrupting visual continuity to induce biased distributions based on language priors and statistical biases, then contrasts these disturbed distributions with the original ones to eliminate object hallucination-inducing factors. Comprehensive experiments across several hallucination benchmarks demonstrate the effectiveness of our method in multiple MLLMs.