Scholarlink

<p>In this paper, we aim to enhance the training of physics-informed neural networks by introducing the preconditioned optimizer—a quasi-Newton method for nonconvex optimization that utilizes Hessian approximations to dynamically capture the curvature of the loss function. Importantly, our preconditioned method has a slightly greater computational cost due to extra computations, but it uses the same amount of memory as adaptive and super-linear order optimization methods for updating and storing the Hessian approximation. The nonlinear preconditioning is constructed by utilizing the Broyden class quasi-Newton updating formulae where the adaptive strategy for accelerating the convergence of proposed method are used. The new method achieves a superlinear rate of convergence, with the preconditioning strategy adapting dynamically due to the introduction of a novel function that enhances the optimization process. Through the series of numerical experiments, we demonstrate that the preconditioned optimizers significantly outperform the standard optimizers, offering improved convergence rates and, in contrast to popular optimizers like Adam and L-BFGS, more precise answers to the underlying partial differential equations.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A preconditioned quasi-newton optimizer for efficient training of PINNs

  • 期刊论文

  • 发表日期：

    2025年9月4日

查看全文
  • Shahbaz Ahmad,
  • Muhammad Israr

摘要

In this paper, we aim to enhance the training of physics-informed neural networks by introducing the preconditioned optimizer—a quasi-Newton method for nonconvex optimization that utilizes Hessian approximations to dynamically capture the curvature of the loss function. Importantly, our preconditioned method has a slightly greater computational cost due to extra computations, but it uses the same amount of memory as adaptive and super-linear order optimization methods for updating and storing the Hessian approximation. The nonlinear preconditioning is constructed by utilizing the Broyden class quasi-Newton updating formulae where the adaptive strategy for accelerating the convergence of proposed method are used. The new method achieves a superlinear rate of convergence, with the preconditioning strategy adapting dynamically due to the introduction of a novel function that enhances the optimization process. Through the series of numerical experiments, we demonstrate that the preconditioned optimizers significantly outperform the standard optimizers, offering improved convergence rates and, in contrast to popular optimizers like Adam and L-BFGS, more precise answers to the underlying partial differential equations.