Multimodal knowledge graph entity alignment aims to identify cross-graph equivalent entities by integrating multi-source heterogeneous information such as images, attributes, and relationships, so as to achieve semantic alignment of knowledge graphs. To address the problems of insufficient modality fusion and inadequate utilization of structural information in existing multimodal methods, this paper proposes a multimodal entity alignment model HSE-GFN with Hierarchical Structure Encoder (HSE) and Gated Fusion Network (GFN). The structural feature extraction of HSE-GFN is based on the Sparse Random Walk Encoder (SRWE) design, which captures multi-scale structural information through multi-layer random walks and employs attention mechanisms to aggregate features at different levels, thereby effectively capturing the hierarchical structural relationships between entities. GFN adopts a three-layer gating architecture (node-level dynamic gating, modality-level cross-gating, and graph-level fusion gating) to fuse cross-modal feature interactions, adaptively integrating heterogeneous modality features through a dynamic feature fusion mechanism. Specifically, node-level gating enables dynamic fusion of different modality feature pairs, modality-level gating handles cross-relationships between multiple modalities, and graph-level gating is responsible for the final fusion of global features. We conducted comparative and ablation experiments on the public datasets DBP15K and FB15K-DB15K/YAGO15K, and the results demonstrate that our method significantly outperforms existing benchmark methods across various metrics and effectively enhances the model’s alignment capability in knowledge graphs.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

HSE-GFN: Multimodal Knowledge Graph Entity Alignment via Hierarchical Structure Encoding and Gated Fusion

  • Fuwen Tang,
  • Yanhui Zhu,
  • Zhixuan Zhang,
  • Fangteng Man,
  • Xujian Ying,
  • Hao Chen,
  • Zhipeng Gan

摘要

Multimodal knowledge graph entity alignment aims to identify cross-graph equivalent entities by integrating multi-source heterogeneous information such as images, attributes, and relationships, so as to achieve semantic alignment of knowledge graphs. To address the problems of insufficient modality fusion and inadequate utilization of structural information in existing multimodal methods, this paper proposes a multimodal entity alignment model HSE-GFN with Hierarchical Structure Encoder (HSE) and Gated Fusion Network (GFN). The structural feature extraction of HSE-GFN is based on the Sparse Random Walk Encoder (SRWE) design, which captures multi-scale structural information through multi-layer random walks and employs attention mechanisms to aggregate features at different levels, thereby effectively capturing the hierarchical structural relationships between entities. GFN adopts a three-layer gating architecture (node-level dynamic gating, modality-level cross-gating, and graph-level fusion gating) to fuse cross-modal feature interactions, adaptively integrating heterogeneous modality features through a dynamic feature fusion mechanism. Specifically, node-level gating enables dynamic fusion of different modality feature pairs, modality-level gating handles cross-relationships between multiple modalities, and graph-level gating is responsible for the final fusion of global features. We conducted comparative and ablation experiments on the public datasets DBP15K and FB15K-DB15K/YAGO15K, and the results demonstrate that our method significantly outperforms existing benchmark methods across various metrics and effectively enhances the model’s alignment capability in knowledge graphs.