<p>Considering demands of real-time status monitoring, due to complex internal environment and unprocurable mass sensors involvement, new approaches for field reconstruction with sparse sensors have always been a hot topic in related works. Most current studies focus on mapping relationships of the data while ignore the local connections among the physical system, which may cause the loss of potential reconstruction methods. To be specific, the neighborhood information, especially graph information, contains some key features which can greatly improve the reconstruction. Motivated by this idea, the graph-based methods have been introduced to the dynamical systems. By representing the discretized physical field as a graph, graph neural networks can naturally capture both local neighborhood features and structural priors, making them well-suited for field reconstruction. However, standard graph attention networks incur <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\mathcal {O}(n^2)\)</EquationSource> </InlineEquation> computational and memory complexity due to the explicit computation of the full <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(n \times n\)</EquationSource> </InlineEquation> attention score, which severely limits the scalability to large-scale systems with tens of thousands of nodes. An efficient graph attention network based on low rank and linear attention is proposed to tackle this quadratic bottleneck in this paper. After modeling the system as a mathematical graph, the proposed network extracts the hidden information by two branches of feature and adjacent connection, which include matrix factorization and linear normalization respectively. Then the reconstruction could be obtained by fusing the features of two branches. With this elaborate structure, the network inherits the merit of high accuracy and avoids the appearance of memory and time consuming matrices, reducing the overall complexity from <InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(\mathcal {O}(n^2)\)</EquationSource> </InlineEquation> to <InlineEquation ID="IEq4"> <EquationSource Format="TEX">\(\mathcal {O}(n \times r)\)</EquationSource> </InlineEquation>, where the low-rank dimension <InlineEquation ID="IEq5"> <EquationSource Format="TEX">\(r \ll n\)</EquationSource> </InlineEquation>. The results of the comparison experiments with baseline methods show the superiority of the proposed method that it has the lowest reconstruction error in almost all the cases of three systems with an average improvement of 6<InlineEquation ID="IEq6"> <EquationSource Format="TEX">\(\%\)</EquationSource> </InlineEquation> in MAE and 3<InlineEquation ID="IEq7"> <EquationSource Format="TEX">\(\%\)</EquationSource> </InlineEquation> in the <InlineEquation ID="IEq8"> <EquationSource Format="TEX">\(l_2\)</EquationSource> </InlineEquation>-norm relative error. Moreover, the ablation experiments of model size and inference time validate the generalization and practicability of the proposed network in some large-scale reconstructions of physical systems.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A novel graph attention network based on low rank factorization and linear attention for field reconstruction

  • Qiao Li,
  • Congcong Jin,
  • Si Wang,
  • Sihan Zhang,
  • Weile Xu,
  • Xingchen Li

摘要

Considering demands of real-time status monitoring, due to complex internal environment and unprocurable mass sensors involvement, new approaches for field reconstruction with sparse sensors have always been a hot topic in related works. Most current studies focus on mapping relationships of the data while ignore the local connections among the physical system, which may cause the loss of potential reconstruction methods. To be specific, the neighborhood information, especially graph information, contains some key features which can greatly improve the reconstruction. Motivated by this idea, the graph-based methods have been introduced to the dynamical systems. By representing the discretized physical field as a graph, graph neural networks can naturally capture both local neighborhood features and structural priors, making them well-suited for field reconstruction. However, standard graph attention networks incur \(\mathcal {O}(n^2)\) computational and memory complexity due to the explicit computation of the full \(n \times n\) attention score, which severely limits the scalability to large-scale systems with tens of thousands of nodes. An efficient graph attention network based on low rank and linear attention is proposed to tackle this quadratic bottleneck in this paper. After modeling the system as a mathematical graph, the proposed network extracts the hidden information by two branches of feature and adjacent connection, which include matrix factorization and linear normalization respectively. Then the reconstruction could be obtained by fusing the features of two branches. With this elaborate structure, the network inherits the merit of high accuracy and avoids the appearance of memory and time consuming matrices, reducing the overall complexity from \(\mathcal {O}(n^2)\) to \(\mathcal {O}(n \times r)\) , where the low-rank dimension \(r \ll n\) . The results of the comparison experiments with baseline methods show the superiority of the proposed method that it has the lowest reconstruction error in almost all the cases of three systems with an average improvement of 6 \(\%\) in MAE and 3 \(\%\) in the \(l_2\) -norm relative error. Moreover, the ablation experiments of model size and inference time validate the generalization and practicability of the proposed network in some large-scale reconstructions of physical systems.