Multimodal Named Entity Recognition (MNER) aims to en-hance entity recognition accuracy by incorporating visual information to complement textual data. Existing approaches have made notable progress by introducing multi-granular visual features to strengthen cross-modal semantic alignment. However, they generally lack effective filtering of feature relevance, which often leads to the inclusion of excessive irrel-evant features and a consequent decline in model performance. To address this limitation, we propose a Multi-granular Feature Selection Fusion (MGFS) method that progressively filters and integrates visual features through three stages: overview inspection, detailed inspection, and feature decision. MGFS combines cross-modal attention, adaptive gating mechanisms, and a dynamic feature selection network guided by uncertainty estimation to effectively filter out irrelevant visual information. Experimental results on two public datasets fully validate the effectiveness and superiority of our proposed method. Specifically, MGFS not only outperforms conventional multimodal approaches but also surpasses recent multimodal frameworks integrated with large language models (LLMs), achieving superior overall performance.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multi-granular Feature Selection Fusion Method for Multimodal Named Entity Recognition

  • Guohui Ding,
  • Tengyu Fan,
  • Chufei Wang

摘要

Multimodal Named Entity Recognition (MNER) aims to en-hance entity recognition accuracy by incorporating visual information to complement textual data. Existing approaches have made notable progress by introducing multi-granular visual features to strengthen cross-modal semantic alignment. However, they generally lack effective filtering of feature relevance, which often leads to the inclusion of excessive irrel-evant features and a consequent decline in model performance. To address this limitation, we propose a Multi-granular Feature Selection Fusion (MGFS) method that progressively filters and integrates visual features through three stages: overview inspection, detailed inspection, and feature decision. MGFS combines cross-modal attention, adaptive gating mechanisms, and a dynamic feature selection network guided by uncertainty estimation to effectively filter out irrelevant visual information. Experimental results on two public datasets fully validate the effectiveness and superiority of our proposed method. Specifically, MGFS not only outperforms conventional multimodal approaches but also surpasses recent multimodal frameworks integrated with large language models (LLMs), achieving superior overall performance.