Scholarlink

<p>Named Entity Recognition (NER) and Relation Extraction (RE) are key tasks for knowledge graph construction and social media analysis. Most existing approaches address these tasks independently and rely solely on textual information, which is often insufficient for social media content where images provide important contextual cues. To address this limitation, we propose a multimodal framework that integrates textual and visual information for both NER and RE. Our approach combines BERT-based textual representations with ResNet-extracted visual features. For Multimodal Named Entity Recognition (MNER), a BiLSTM–CRF architecture is employed to support structured sequence labeling, along with a relevance-driven image weighting mechanism to regulate visual contributions. For Multimodal Relation Extraction (MRE), a prompt-based multimodal integration strategy with gated feature refinement is used to model cross-modal dependencies. The proposed framework demonstrates the effectiveness of multimodal fusion for both NER and RE tasks. To support unified multimodal evaluation, we construct a benchmark dataset by reformulating MNRE into a Twitter-style sequence labeling format while preserving relation annotations. This dataset enables NER-only, RE-only, and pipeline-based NER-to-RE evaluation settings, and provides a foundation for future exploration of joint multimodal NER–RE modeling.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

NER2RE: A Multimodal Benchmark for Named Entity Recognition and Relation Extraction in Social Media

  • 期刊论文

  • 发表日期：

    2026年5月27日

查看全文
  • M. Saritha,
  • T. Vaishnavi,
  • J. S. Vishvasri

摘要

Named Entity Recognition (NER) and Relation Extraction (RE) are key tasks for knowledge graph construction and social media analysis. Most existing approaches address these tasks independently and rely solely on textual information, which is often insufficient for social media content where images provide important contextual cues. To address this limitation, we propose a multimodal framework that integrates textual and visual information for both NER and RE. Our approach combines BERT-based textual representations with ResNet-extracted visual features. For Multimodal Named Entity Recognition (MNER), a BiLSTM–CRF architecture is employed to support structured sequence labeling, along with a relevance-driven image weighting mechanism to regulate visual contributions. For Multimodal Relation Extraction (MRE), a prompt-based multimodal integration strategy with gated feature refinement is used to model cross-modal dependencies. The proposed framework demonstrates the effectiveness of multimodal fusion for both NER and RE tasks. To support unified multimodal evaluation, we construct a benchmark dataset by reformulating MNRE into a Twitter-style sequence labeling format while preserving relation annotations. This dataset enables NER-only, RE-only, and pipeline-based NER-to-RE evaluation settings, and provides a foundation for future exploration of joint multimodal NER–RE modeling.