Fine-grained image recognition tasks necessitate models capable of distinguishing visually similar categories, presenting a challenge for current multimodal models. To enhance models’ performance and interpretability in such tasks, this paper proposes a multimodal reasoning framework that integrates dynamic visual attention mechanisms with chain-like language reasoning mechanisms. The proposed method first employs a learnable dynamic visual attention module to iteratively focus on key local regions during the reasoning process. Subsequently, a graph neural network encodes structured visual scene graphs based on object relationships. Finally, a large language model drives multi-step chain reasoning, generating step-by-step reasoning texts and producing discriminative results. Experimental results on multiple fine-grained classification benchmarks demonstrate that this approach achieves a 3%–5% improvement in Top-1 and Top-5 accuracy, effectively reducing modality bias and inconsistencies during the model's reasoning process.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

From Ambiguity to Precision: Multimodal Chain Reasoning with Dynamic Visual Grounding for Fine-Grained Recognition

  • Jiarui Xie,
  • Jie Yang

摘要

Fine-grained image recognition tasks necessitate models capable of distinguishing visually similar categories, presenting a challenge for current multimodal models. To enhance models’ performance and interpretability in such tasks, this paper proposes a multimodal reasoning framework that integrates dynamic visual attention mechanisms with chain-like language reasoning mechanisms. The proposed method first employs a learnable dynamic visual attention module to iteratively focus on key local regions during the reasoning process. Subsequently, a graph neural network encodes structured visual scene graphs based on object relationships. Finally, a large language model drives multi-step chain reasoning, generating step-by-step reasoning texts and producing discriminative results. Experimental results on multiple fine-grained classification benchmarks demonstrate that this approach achieves a 3%–5% improvement in Top-1 and Top-5 accuracy, effectively reducing modality bias and inconsistencies during the model's reasoning process.