Scholarlink

<p>In recent years, graph neural networks (GNNs) have emerged as a promising approach for classifying non-Euclidean structural data. However, the practical implementation of GNNs faces challenges related to their limited scalability due to the presence of multi-hop data dependencies. In order to tackle this issue, existing methods have employed teacher GNNs to generate labels, which are then used to train multilayer perceptrons (MLPs) based solely on node features, without considering any structural information. However, these methods primarily focus on the soft labels generated by the teacher GNNs, leading to suboptimal performance because they disregard significant features of the teacher GNNs. Additionally, since the structural information of the graph is omitted in the input of MLPs, it can be more susceptible to the influence of erroneous features. To address these limitations, this paper proposes a novel framework that incorporates a new distillation strategy to integrate soft feature similarity into MLPs, while also utilizing contrastive learning to enhance the training of student MLPs. Our model has accuracy, averaged over seven public datasets, 5.21% higher than state-of-the-art (SOTA) methods, and even 2.49% higher than teacher GNNs over five standard scaled datasets. At the same time, its inference time is only 1.17% of comparable GNNs.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Contrastive Enhanced Knowledge Distillation for Learning MLPs on GNNs

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年7月16日

查看全文
  • Wei Hu,
  • Juhua Pu,
  • Xiaolan Tang,
  • Xingwu Liu

摘要

In recent years, graph neural networks (GNNs) have emerged as a promising approach for classifying non-Euclidean structural data. However, the practical implementation of GNNs faces challenges related to their limited scalability due to the presence of multi-hop data dependencies. In order to tackle this issue, existing methods have employed teacher GNNs to generate labels, which are then used to train multilayer perceptrons (MLPs) based solely on node features, without considering any structural information. However, these methods primarily focus on the soft labels generated by the teacher GNNs, leading to suboptimal performance because they disregard significant features of the teacher GNNs. Additionally, since the structural information of the graph is omitted in the input of MLPs, it can be more susceptible to the influence of erroneous features. To address these limitations, this paper proposes a novel framework that incorporates a new distillation strategy to integrate soft feature similarity into MLPs, while also utilizing contrastive learning to enhance the training of student MLPs. Our model has accuracy, averaged over seven public datasets, 5.21% higher than state-of-the-art (SOTA) methods, and even 2.49% higher than teacher GNNs over five standard scaled datasets. At the same time, its inference time is only 1.17% of comparable GNNs.