Scholarlink

<p>Nonconvex and nonsmooth composite optimization problems with linear constraints have attracted widespread attention in artificial intelligence and computer science due to their extensive applications in machine learning. Variance-reduced stochastic ADMM algorithms are extensively employed to address these problems, with most existing methods relying on the classical SVRG double-loop framework. In this study, we introduce the SVRRM-ADMM algorithm, which leverages a novel stochastic variance-reduced recursive momentum (SVRRM) estimator constructed on the loopless-SVRG framework and integrates it with the Stochastic ADMM (SADMM). The proposed algorithm facilitates implementation, requires fewer tuning parameters, and retains equivalent theoretical properties. We demonstrate that SVRRM-ADMM converges to a stationary solution without assuming bounded variance. Moreover, we extend the SVRRM-ADMM algorithm by incorporating acceleration techniques to yield the ASVRRM-ADMM. We establish that both SVRRM-ADMM and ASVRRM-ADMM achieve a worst-case convergence rate of <i>O</i>(1/<i>T</i>), where <i>T</i> denotes the number of iterations. Under the additional Kurdyka-Lojasiewicz (KL) property assumption, we further show that the sequences generated by both algorithms have finite expected length and attain different convergence rates determined by the KL exponent. Finally, numerical experiments validate the effectiveness of the proposed algorithms.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Stochastic ADMM with variance-reduced recursive momentum and its accelerated variant for nonconvex nonsmooth optimization

  • 期刊论文

  • 发表日期：

    2026年4月18日

查看全文
  • Feiyu Long,
  • Congying Han,
  • Tiande Guo,
  • Shichen Liao

摘要

Nonconvex and nonsmooth composite optimization problems with linear constraints have attracted widespread attention in artificial intelligence and computer science due to their extensive applications in machine learning. Variance-reduced stochastic ADMM algorithms are extensively employed to address these problems, with most existing methods relying on the classical SVRG double-loop framework. In this study, we introduce the SVRRM-ADMM algorithm, which leverages a novel stochastic variance-reduced recursive momentum (SVRRM) estimator constructed on the loopless-SVRG framework and integrates it with the Stochastic ADMM (SADMM). The proposed algorithm facilitates implementation, requires fewer tuning parameters, and retains equivalent theoretical properties. We demonstrate that SVRRM-ADMM converges to a stationary solution without assuming bounded variance. Moreover, we extend the SVRRM-ADMM algorithm by incorporating acceleration techniques to yield the ASVRRM-ADMM. We establish that both SVRRM-ADMM and ASVRRM-ADMM achieve a worst-case convergence rate of O(1/T), where T denotes the number of iterations. Under the additional Kurdyka-Lojasiewicz (KL) property assumption, we further show that the sequences generated by both algorithms have finite expected length and attain different convergence rates determined by the KL exponent. Finally, numerical experiments validate the effectiveness of the proposed algorithms.