Scholarlink

<p>Knowledge graph embedding (KGE) is critical for efficient knowledge reasoning and completion. Existing KGE models are primarily designed for static knowledge graphs, often utilizing knowledge distillation to enhance inference efficiency and reduce storage requirements. However, dynamic updates to knowledge graphs significantly increase training costs and lead to catastrophic forgetting of previous knowledge. To address this challenge, this paper proposes a KGE framework (CI-KGE) that combines cascade and incremental knowledge distillation for effective embedding in dynamic knowledge graphs. First, CI-KGE employs cascade distillation to gradually reduce the dimensional gap between high-dimensional teacher and low-dimensional student models. By transferring essential features during each iteration, the framework ensures resource efficiency while maintaining strong inference performance. To accommodate dynamic updates, the framework introduces an incremental distillation mechanism that organizes new knowledge into hierarchical layers based on graph structure. It progressively integrates knowledge both within and across layers, ensuring preservation of the graph’s semantic structure. Additionally, CI-KGE incorporates a three-stage training strategy that balances optimization between hard and soft labels, and adopts hierarchical learning to prioritize newly introduced knowledge. This enhances model adaptability to evolving knowledge graphs. Experimental results demonstrate that CI-KGE performs well in link prediction tasks compared to state-of-the-art methods, while maintaining stable embedding performance during incremental updates. The framework is designed for high-performance computing environments, with a parallel computing architecture designed for scalable deployment on multi-GPU configurations through data parallelism, pipeline parallelism, and distributed memory management. The framework’s modular design enables flexible resource allocation based on knowledge graph scale and HPC system specifications. The complete cascade distillation across three stages can be parallelized through pipeline execution, and incremental updates benefit from the compressed model size, requiring substantially less time than full model retraining. The framework is architected for scalable deployment in supercomputing environments, with distributed memory management designed to support large-scale knowledge graph processing.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A cascade knowledge distillation framework with three-stage training for dynamic knowledge graph embedding

  • 期刊论文

  • 发表日期：

    2025年12月25日

查看全文
  • Heng Zhang,
  • Shuguang Zhang

摘要

Knowledge graph embedding (KGE) is critical for efficient knowledge reasoning and completion. Existing KGE models are primarily designed for static knowledge graphs, often utilizing knowledge distillation to enhance inference efficiency and reduce storage requirements. However, dynamic updates to knowledge graphs significantly increase training costs and lead to catastrophic forgetting of previous knowledge. To address this challenge, this paper proposes a KGE framework (CI-KGE) that combines cascade and incremental knowledge distillation for effective embedding in dynamic knowledge graphs. First, CI-KGE employs cascade distillation to gradually reduce the dimensional gap between high-dimensional teacher and low-dimensional student models. By transferring essential features during each iteration, the framework ensures resource efficiency while maintaining strong inference performance. To accommodate dynamic updates, the framework introduces an incremental distillation mechanism that organizes new knowledge into hierarchical layers based on graph structure. It progressively integrates knowledge both within and across layers, ensuring preservation of the graph’s semantic structure. Additionally, CI-KGE incorporates a three-stage training strategy that balances optimization between hard and soft labels, and adopts hierarchical learning to prioritize newly introduced knowledge. This enhances model adaptability to evolving knowledge graphs. Experimental results demonstrate that CI-KGE performs well in link prediction tasks compared to state-of-the-art methods, while maintaining stable embedding performance during incremental updates. The framework is designed for high-performance computing environments, with a parallel computing architecture designed for scalable deployment on multi-GPU configurations through data parallelism, pipeline parallelism, and distributed memory management. The framework’s modular design enables flexible resource allocation based on knowledge graph scale and HPC system specifications. The complete cascade distillation across three stages can be parallelized through pipeline execution, and incremental updates benefit from the compressed model size, requiring substantially less time than full model retraining. The framework is architected for scalable deployment in supercomputing environments, with distributed memory management designed to support large-scale knowledge graph processing.