<p>Graph Convolutional Networks (GCNs) have been widely adopted to learn structured data that denotes structural connections between entities. However, over-smoothing has become a common phenomenon in GCNs, resulting in node representations in different classes are indistinguishable. Inspired by recent works, graph disentanglement, which can learn disentangled latent factors behind the input data and generate latent representations corresponding to latent factors, is adopted to relieve over-smoothing. Although graph disentanglement is promising, it still faces some challenges. (1) How to obtain disentangled latent representations useful for both latent factors and downstream tasks? (2) How to enhance the diversity and independence of latent representations? In this paper, we propose a novel <Emphasis Type="BoldUnderline">GCN</Emphasis> model based on Graph <Emphasis Type="BoldUnderline">D</Emphasis>isentanglement with Latent <Emphasis Type="BoldUnderline">B</Emphasis>ottlenecks (<b>DBGCN</b>) to boost the diversity of node representations and relieve the over-smoothing problem. To learn disentangled node representations that are beneficial to both latent factors and downstream tasks, we propose to identify latent bottlenecks using the information bottleneck, which offers a critical principle for good representation learning. To boost the diversity of latent representations and the independence between them, we design a generative model to search for specific latent bottlenecks that constrain information flow from the input using specific latent factors, and to derive distributions of latent representations. Experimental results on various real-world graphs demonstrate that the proposed model can relieve the over-smoothing problem effectively and achieve significant performance gains. It also suggests that enhancing the diversity of latent representations is helpful to relieve over-smoothing.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Relieving the Over-Smoothing Problem using Graph Disentanglement with Latent Bottlenecks

  • Guoqiang Zhou,
  • Wenzhen Liu,
  • Kexi Xu,
  • Guilan Dai,
  • Qian Liu,
  • Huan Wang,
  • Jun Shen

摘要

Graph Convolutional Networks (GCNs) have been widely adopted to learn structured data that denotes structural connections between entities. However, over-smoothing has become a common phenomenon in GCNs, resulting in node representations in different classes are indistinguishable. Inspired by recent works, graph disentanglement, which can learn disentangled latent factors behind the input data and generate latent representations corresponding to latent factors, is adopted to relieve over-smoothing. Although graph disentanglement is promising, it still faces some challenges. (1) How to obtain disentangled latent representations useful for both latent factors and downstream tasks? (2) How to enhance the diversity and independence of latent representations? In this paper, we propose a novel GCN model based on Graph Disentanglement with Latent Bottlenecks (DBGCN) to boost the diversity of node representations and relieve the over-smoothing problem. To learn disentangled node representations that are beneficial to both latent factors and downstream tasks, we propose to identify latent bottlenecks using the information bottleneck, which offers a critical principle for good representation learning. To boost the diversity of latent representations and the independence between them, we design a generative model to search for specific latent bottlenecks that constrain information flow from the input using specific latent factors, and to derive distributions of latent representations. Experimental results on various real-world graphs demonstrate that the proposed model can relieve the over-smoothing problem effectively and achieve significant performance gains. It also suggests that enhancing the diversity of latent representations is helpful to relieve over-smoothing.