Scholarlink

<p>Chinese painting is an important form of cultural heritage. Its digital resources, including images and descriptive texts, contain rich yet unstructured information. Multimodal named entity recognition (MNER) is crucial for knowledge extraction but faces challenges including the lack of domain-specific datasets, limited integration of external knowledge, and semantic gaps between modalities. To address these issues, we construct CP-MNER, an MNER dataset for Chinese painting, and establish standardized baselines. We further propose MFKA, a multi-path fusion framework with knowledge augmentation. MFKA generates text-aware visual representations through cross-modal attention and incorporates external knowledge through a two-stage process using multimodal large language models. A multi-path complementary fusion module then integrates textual, visual, and knowledge-enhanced representations for multimodal semantic alignment. Experimental results demonstrate that MFKA achieves state-of-the-art performance on CP-MNER.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A multi-path fusion with knowledge augmentation framework for multimodal NER in Chinese painting

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年4月18日

查看全文
  • Jing Wan,
  • Siyun Chen,
  • Qingyang Zeng,
  • Rumei Wang

摘要

Chinese painting is an important form of cultural heritage. Its digital resources, including images and descriptive texts, contain rich yet unstructured information. Multimodal named entity recognition (MNER) is crucial for knowledge extraction but faces challenges including the lack of domain-specific datasets, limited integration of external knowledge, and semantic gaps between modalities. To address these issues, we construct CP-MNER, an MNER dataset for Chinese painting, and establish standardized baselines. We further propose MFKA, a multi-path fusion framework with knowledge augmentation. MFKA generates text-aware visual representations through cross-modal attention and incorporates external knowledge through a two-stage process using multimodal large language models. A multi-path complementary fusion module then integrates textual, visual, and knowledge-enhanced representations for multimodal semantic alignment. Experimental results demonstrate that MFKA achieves state-of-the-art performance on CP-MNER.