From Ambiguity to Precision: Multimodal Chain Reasoning with Dynamic Visual Grounding for Fine-Grained Recognition
摘要
Fine-grained image recognition tasks necessitate models capable of distinguishing visually similar categories, presenting a challenge for current multimodal models. To enhance models’ performance and interpretability in such tasks, this paper proposes a multimodal reasoning framework that integrates dynamic visual attention mechanisms with chain-like language reasoning mechanisms. The proposed method first employs a learnable dynamic visual attention module to iteratively focus on key local regions during the reasoning process. Subsequently, a graph neural network encodes structured visual scene graphs based on object relationships. Finally, a large language model drives multi-step chain reasoning, generating step-by-step reasoning texts and producing discriminative results. Experimental results on multiple fine-grained classification benchmarks demonstrate that this approach achieves a 3%–5% improvement in Top-1 and Top-5 accuracy, effectively reducing modality bias and inconsistencies during the model's reasoning process.