Scholarlink

<p>Nonsmooth and nonconvex optimization problems are central to many machine learning applications, such as matrix factorization, tensor decomposition, and deep learning. Existing PALM-type algorithms and their inertial variants have shown practical success, yet they remain limited by restrictive parameter conditions and incomplete theoretical guarantees. In this paper, we propose an improved inertial stochastic proximal alternating linearized minimization algorithm (IiSPALM). The method introduces a novel double-inertial mechanism applied both before and after each block update, while avoiding the rigidity of nonzero inertial parameters required. By combining this design with variance reduced stochastic gradient estimators, we establish the theoretical results: IiSPALM achieves an iteration complexity of <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\mathcal {O}(\varepsilon ^{-2})\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi mathvariant="script">O</mi> <mo stretchy="false">(</mo> <msup> <mi>ε</mi> <mrow> <mo>-</mo> <mn>2</mn> </mrow> </msup> <mo stretchy="false">)</mo> </mrow> </math></EquationSource> </InlineEquation> for finding an <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(\varepsilon \)</EquationSource> <EquationSource Format="MATHML"><math> <mi>ε</mi> </math></EquationSource> </InlineEquation>-stationary point, guarantees linear convergence under the Polyak–Łojasiewicz condition, and ensures global convergence via the Kurdyka–Łojasiewicz property. Numerical experiments on nonnegative sparse matrix decomposition, tensor CP decomposition, and proximal neural networks demonstrate that IiSPALM consistently outperforms state-of-the-art deterministic and stochastic PALM-type algorithms. These results confirm that IiSPALM bridges the theoretical and practical gaps left by previous methods, offering a flexible and efficient framework for large-scale nonconvex optimization. The code is available at <a href="https://github.com/nothing2wang/IiSPALM">https://github.com/nothing2wang/IiSPALM</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

An improved inertial stochastic proximal alternating linearized minimization for nonconvex optimization in machine learning

  • 期刊论文

  • 发表日期：

    2026年7月22日

查看全文
  • Qingsong Wang,
  • Deren Han

摘要

Nonsmooth and nonconvex optimization problems are central to many machine learning applications, such as matrix factorization, tensor decomposition, and deep learning. Existing PALM-type algorithms and their inertial variants have shown practical success, yet they remain limited by restrictive parameter conditions and incomplete theoretical guarantees. In this paper, we propose an improved inertial stochastic proximal alternating linearized minimization algorithm (IiSPALM). The method introduces a novel double-inertial mechanism applied both before and after each block update, while avoiding the rigidity of nonzero inertial parameters required. By combining this design with variance reduced stochastic gradient estimators, we establish the theoretical results: IiSPALM achieves an iteration complexity of \(\mathcal {O}(\varepsilon ^{-2})\) O ( ε - 2 ) for finding an \(\varepsilon \) ε -stationary point, guarantees linear convergence under the Polyak–Łojasiewicz condition, and ensures global convergence via the Kurdyka–Łojasiewicz property. Numerical experiments on nonnegative sparse matrix decomposition, tensor CP decomposition, and proximal neural networks demonstrate that IiSPALM consistently outperforms state-of-the-art deterministic and stochastic PALM-type algorithms. These results confirm that IiSPALM bridges the theoretical and practical gaps left by previous methods, offering a flexible and efficient framework for large-scale nonconvex optimization. The code is available at https://github.com/nothing2wang/IiSPALM.