<p>Separable optimization models are foundational in machine learning, signal processing, and system identification. A key to their efficient identification lies in resolving the coupling between linear and nonlinear parameters during the optimization process. Yet, real-world applications often impose priors like sparsity or total variation (TV) regularization, introducing non-convexity and non-smoothness that significantly complicate this decoupling and pose serious challenges for standard solvers. To address this, we introduce a novel inexact variable projection algorithm grounded in an inexact decoupling framework, termed as InVP(Inexact Variable Projection). Unlike classical methods, InVP begins with just a few iterative steps to approximate the optimal linear parameters, significantly reducing the computational burden of exact solutions. It then introduces an inexact decoupling strategy, effectively resolving the coupling between linear and nonlinear parameters when updating the nonlinear ones. Moreover, our experiments theoretically characterize a uniform upper bound on the error induced by the inexact strategy in the simplified Jacobian of the objective function, based on this bound, establish global convergence and local convergence rate results for the algorithm. Numerical experiments on data fitting, nonlinear time-series analysis, and tomographic reconstruction further demonstrate the effectiveness of the proposed InVP algorithm.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

An efficient approach for addressing non-smooth separable optimization problems

  • Guang-Yong Chen,
  • Yu-Lin Tang,
  • Xiang-Xiang Su,
  • Peng Xue

摘要

Separable optimization models are foundational in machine learning, signal processing, and system identification. A key to their efficient identification lies in resolving the coupling between linear and nonlinear parameters during the optimization process. Yet, real-world applications often impose priors like sparsity or total variation (TV) regularization, introducing non-convexity and non-smoothness that significantly complicate this decoupling and pose serious challenges for standard solvers. To address this, we introduce a novel inexact variable projection algorithm grounded in an inexact decoupling framework, termed as InVP(Inexact Variable Projection). Unlike classical methods, InVP begins with just a few iterative steps to approximate the optimal linear parameters, significantly reducing the computational burden of exact solutions. It then introduces an inexact decoupling strategy, effectively resolving the coupling between linear and nonlinear parameters when updating the nonlinear ones. Moreover, our experiments theoretically characterize a uniform upper bound on the error induced by the inexact strategy in the simplified Jacobian of the objective function, based on this bound, establish global convergence and local convergence rate results for the algorithm. Numerical experiments on data fitting, nonlinear time-series analysis, and tomographic reconstruction further demonstrate the effectiveness of the proposed InVP algorithm.