<p>We present a damped proximal augmented Lagrangian method (DPALM) for solving problems with a weakly-convex objective and convex linear/non-linear constraints. Instead of taking a full stepsize, DPALM adopts a (possibly) damped dual stepsize. We show that DPALM can produce a (near) <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(\varepsilon \)</EquationSource> <EquationSource Format="MATHML"><math> <mi>ε</mi> </math></EquationSource> </InlineEquation>-KKT point within <InlineEquation ID="IEq3"> <EquationSource Format="TEX">\({\mathcal {O}}(\varepsilon ^{-2})\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mi mathvariant="script">O</mi> <mo stretchy="false">(</mo> <msup> <mi>ε</mi> <mrow> <mo>-</mo> <mn>2</mn> </mrow> </msup> <mo stretchy="false">)</mo> </mrow> </math></EquationSource> </InlineEquation> outer iterations if each DPALM subproblem is solved to a proper accuracy. In addition, we establish overall oracle complexity (i.e., total number of evaluations of function values and (sub)gradients) of DPALM when the objective is either a regularized smooth function or in a regularized compositional form. For the former case, DPALM achieves a complexity of <InlineEquation ID="IEq4"> <EquationSource Format="TEX">\(\widetilde{\mathcal {O}}\left( \varepsilon ^{-2.5} \right) \)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mover accent="true"> <mi mathvariant="script">O</mi> <mo stretchy="true">~</mo> </mover> <mfenced close=")" open="("> <msup> <mi>ε</mi> <mrow> <mo>-</mo> <mn>2.5</mn> </mrow> </msup> </mfenced> </mrow> </math></EquationSource> </InlineEquation> to produce an <InlineEquation ID="IEq5"> <EquationSource Format="TEX">\(\varepsilon \)</EquationSource> <EquationSource Format="MATHML"><math> <mi>ε</mi> </math></EquationSource> </InlineEquation>-KKT point by applying an accelerated proximal gradient (APG) method to each DPALM subproblem. For the latter case, the complexity of DPALM is <InlineEquation ID="IEq6"> <EquationSource Format="TEX">\(\widetilde{\mathcal {O}}\left( \varepsilon ^{-3} \right) \)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mover accent="true"> <mi mathvariant="script">O</mi> <mo stretchy="true">~</mo> </mover> <mfenced close=")" open="("> <msup> <mi>ε</mi> <mrow> <mo>-</mo> <mn>3</mn> </mrow> </msup> </mfenced> </mrow> </math></EquationSource> </InlineEquation> to produce a near <InlineEquation ID="IEq7"> <EquationSource Format="TEX">\(\varepsilon \)</EquationSource> <EquationSource Format="MATHML"><math> <mi>ε</mi> </math></EquationSource> </InlineEquation>-KKT point by using an APG method to solve a Moreau-envelope smoothed version of each subproblem. Our outer iteration complexity and the overall complexity either generalize existing best ones from unconstrained or linear-constrained problems to convex-constrained ones, or improve over the best-known results on solving the same-structured problems. Furthermore, numerical experiments on linearly/quadratically constrained non-convex quadratic programs and linear-constrained robust nonlinear least squares are conducted to demonstrate the empirical efficiency of the proposed DPALM over several state-of-the-art methods and a specialized solver. Our code can be accessed at <a href="https://github.com/RPI-OPT/DampedALM">https://github.com/RPI-OPT/DampedALM</a>.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Damped Proximal Augmented Lagrangian Method for weakly-Convex Problems with Convex Constraints

  • Hari Dahal,
  • Wei Liu,
  • Yangyang Xu

摘要

We present a damped proximal augmented Lagrangian method (DPALM) for solving problems with a weakly-convex objective and convex linear/non-linear constraints. Instead of taking a full stepsize, DPALM adopts a (possibly) damped dual stepsize. We show that DPALM can produce a (near) \(\varepsilon \) ε -KKT point within \({\mathcal {O}}(\varepsilon ^{-2})\) O ( ε - 2 ) outer iterations if each DPALM subproblem is solved to a proper accuracy. In addition, we establish overall oracle complexity (i.e., total number of evaluations of function values and (sub)gradients) of DPALM when the objective is either a regularized smooth function or in a regularized compositional form. For the former case, DPALM achieves a complexity of \(\widetilde{\mathcal {O}}\left( \varepsilon ^{-2.5} \right) \) O ~ ε - 2.5 to produce an \(\varepsilon \) ε -KKT point by applying an accelerated proximal gradient (APG) method to each DPALM subproblem. For the latter case, the complexity of DPALM is \(\widetilde{\mathcal {O}}\left( \varepsilon ^{-3} \right) \) O ~ ε - 3 to produce a near \(\varepsilon \) ε -KKT point by using an APG method to solve a Moreau-envelope smoothed version of each subproblem. Our outer iteration complexity and the overall complexity either generalize existing best ones from unconstrained or linear-constrained problems to convex-constrained ones, or improve over the best-known results on solving the same-structured problems. Furthermore, numerical experiments on linearly/quadratically constrained non-convex quadratic programs and linear-constrained robust nonlinear least squares are conducted to demonstrate the empirical efficiency of the proposed DPALM over several state-of-the-art methods and a specialized solver. Our code can be accessed at https://github.com/RPI-OPT/DampedALM.