Scholarlink

<p>In recent years, the inertial extrapolation step has gained significant attention due to its capacity to expedite algorithm convergence. This technology has found widespread application across various algorithms. However, within the domain of machine learning, the utilization of extrapolation technology has yielded limited results. Therefore, we apply it to stochastic optimization algorithms to address non-convex and machine learning problems. By integrating the inertial extrapolation step and the modified Barzilai-Borwein (BB) technique into the SARAH framework, we propose an inertial stochastic recurrence gradient method. This method incorporates both the inertial extrapolation step and the improved BB technique. Through theoretical analysis presented in this paper, we demonstrate that the algorithm converges to a global optimum and analyze the linear convergence rate of the non-convex (<InlineEquation ID="IEq1"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="13042_2024_2524_Article_IEq1.gif" Format="GIF" Height="18" Rendition="HTML" Resolution="72" Type="Linedraw" Width="15" /> </InlineMediaObject> <EquationSource Format="TEX">\(\tilde{\lambda }\)</EquationSource> <EquationSource Format="MATHML"><math> <mover accent="true"> <mi>λ</mi> <mo stretchy="false">~</mo> </mover> </math></EquationSource> </InlineEquation>-gradient-dominated) objective functions. The numerical results obtained from evaluating three widely utilized machine learning problems clearly illustrate the superior performance and practical feasibility of the proposed algorithm.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A stochastic recursive gradient algorithm with inertial extrapolation for non-convex problems and machine learning

  • 期刊论文

  • 发表日期：

    2025年1月9日

查看全文
  • Zhongyu Mo,
  • Chen Ouyang,
  • Hongtruong Pham,
  • Gonglin Yuan

摘要

In recent years, the inertial extrapolation step has gained significant attention due to its capacity to expedite algorithm convergence. This technology has found widespread application across various algorithms. However, within the domain of machine learning, the utilization of extrapolation technology has yielded limited results. Therefore, we apply it to stochastic optimization algorithms to address non-convex and machine learning problems. By integrating the inertial extrapolation step and the modified Barzilai-Borwein (BB) technique into the SARAH framework, we propose an inertial stochastic recurrence gradient method. This method incorporates both the inertial extrapolation step and the improved BB technique. Through theoretical analysis presented in this paper, we demonstrate that the algorithm converges to a global optimum and analyze the linear convergence rate of the non-convex ( \(\tilde{\lambda }\) λ ~ -gradient-dominated) objective functions. The numerical results obtained from evaluating three widely utilized machine learning problems clearly illustrate the superior performance and practical feasibility of the proposed algorithm.