Entropy-regularized multimodal fusion for robust and explainable knowledge graph completion
摘要
Multimodal knowledge graphs (MKGs) serve as critical infrastructure for data mining by integrating structured entities with heterogeneous data, such as text and images. However, real-world MKGs frequently grapple with data sparsity, modality imbalance, and irrelevant cross-modal noise, resulting in optimization instability and poor generalization in MKG completion tasks. To mitigate these challenges, we propose entropy-regularized multimodal fusion (ERMF), a framework designed to achieve robust and explainable representation learning. Specifically, ERMF constructs a unified heterogeneous graph and employs a graph attention network (GAT) to dynamically model context-aware dependencies among structured, textual, and visual features. A context-aware gating network further recalibrates modality contributions, adaptively emphasizing informative signals while suppressing noisy ones. Crucially, to prevent overconfident or biased fusion, we introduce an entropy-regularized term that explicitly enforces a balanced modality distribution, thereby enhancing both robustness and interpretability. Extensive experiments on two benchmark datasets (FB15K-237-IMG and WN18-IMG) demonstrate that ERMF consistently outperforms state-of-the-art models, achieving substantial gains of +2.7% Hits@10 on FB15K-237-IMG and +2.57% Hits@3 on WN18-IMG. Comprehensive empirical evaluations, covering ablation studies, robustness checks, and sensitivity analyses, validate the framework’s stability and reveal that ViT-based encoders yield stronger visual semantics than VGG16, while a single-layer GAT strikes an optimal balance between expressiveness and computational efficiency. ERMF provides a principled, interpretable, and robust solution for multimodal knowledge graph completion under noisy and sparse conditions.