An efficient approach for addressing non-smooth separable optimization problems
摘要
Separable optimization models are foundational in machine learning, signal processing, and system identification. A key to their efficient identification lies in resolving the coupling between linear and nonlinear parameters during the optimization process. Yet, real-world applications often impose priors like sparsity or total variation (TV) regularization, introducing non-convexity and non-smoothness that significantly complicate this decoupling and pose serious challenges for standard solvers. To address this, we introduce a novel inexact variable projection algorithm grounded in an inexact decoupling framework, termed as InVP(Inexact Variable Projection). Unlike classical methods, InVP begins with just a few iterative steps to approximate the optimal linear parameters, significantly reducing the computational burden of exact solutions. It then introduces an inexact decoupling strategy, effectively resolving the coupling between linear and nonlinear parameters when updating the nonlinear ones. Moreover, our experiments theoretically characterize a uniform upper bound on the error induced by the inexact strategy in the simplified Jacobian of the objective function, based on this bound, establish global convergence and local convergence rate results for the algorithm. Numerical experiments on data fitting, nonlinear time-series analysis, and tomographic reconstruction further demonstrate the effectiveness of the proposed InVP algorithm.