Named Entity Recognition (NER) and Relationship Extraction (RE) are critical tasks involved in both information extraction and knowledge graph construction. However, traditional NER and RE methods present limitations when dealing with emergency situations due to the inability to exploit image information. To address this problem, this paper proposes a multimodal Span-based pipeline model, combining ALBEF (ALign the image and text representations BEfore Fusing) for extracting the image features and BERT (Bidirectional Encoder Representations from Transformers) for extracting the text features. By incorporating attention mechanisms, the model can dynamically extract relevant features from texts and images, improving the consistency and accuracy of the entity-relationship extraction. Experimental results show that the proposed method can accurately extract entities and relationships in emergency scenarios, demonstrating its high practical utility.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Emergency Entity and Relationship Extraction Based on a Multimodal Deep Learning Method

  • Peng Wu,
  • Yu Li,
  • Liang Ni

摘要

Named Entity Recognition (NER) and Relationship Extraction (RE) are critical tasks involved in both information extraction and knowledge graph construction. However, traditional NER and RE methods present limitations when dealing with emergency situations due to the inability to exploit image information. To address this problem, this paper proposes a multimodal Span-based pipeline model, combining ALBEF (ALign the image and text representations BEfore Fusing) for extracting the image features and BERT (Bidirectional Encoder Representations from Transformers) for extracting the text features. By incorporating attention mechanisms, the model can dynamically extract relevant features from texts and images, improving the consistency and accuracy of the entity-relationship extraction. Experimental results show that the proposed method can accurately extract entities and relationships in emergency scenarios, demonstrating its high practical utility.