Scholarlink

<p>Bilevel programming has emerged as a valuable tool for hyperparameter selection, a central concern in machine learning. In a recent study by Ye et al. (Math Program <b>198</b>(2):1583–1616, 2023), a value function-based difference of convex algorithm was introduced to address bilevel programs. This approach proves particularly powerful when dealing with scenarios where the lower-level problem exhibits joint convexity in both the upper-level and lower-level variables. Examples of such scenarios include support vector machines and <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(\ell _1\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>ℓ</mi> <mn>1</mn> </msub> </math></EquationSource> </InlineEquation> and <InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(\ell _2\)</EquationSource> <EquationSource Format="MATHML"><math> <msub> <mi>ℓ</mi> <mn>2</mn> </msub> </math></EquationSource> </InlineEquation> regularized regression. In this paper, to expand the scope of possible applications, we substantially weaken the key assumption to only require the lower-level program to be convex for any fixed value of the upper-level program. We present an innovative single-level difference of weakly convex reformulation based on the Moreau envelope of the lower-level problem. We further develop a sequentially convergent Inexact Proximal Difference of Weakly Convex Algorithm (iP-DwCA). To evaluate the effectiveness of the proposed iP-DwCA, we conduct numerical experiments focused on tuning hyperparameters for kernel support vector machines on simulated data.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Moreau Envelope-Based Difference of Weakly Convex Reformulation and Algorithm for Bilevel Programs

  • 期刊论文

  • 发表日期：

    2026年6月27日

查看全文
  • Lucy Gao,
  • Jane J. Ye,
  • Hai-An Yin,
  • Shang-Zhi Zeng,
  • Jin Zhang

摘要

Bilevel programming has emerged as a valuable tool for hyperparameter selection, a central concern in machine learning. In a recent study by Ye et al. (Math Program 198(2):1583–1616, 2023), a value function-based difference of convex algorithm was introduced to address bilevel programs. This approach proves particularly powerful when dealing with scenarios where the lower-level problem exhibits joint convexity in both the upper-level and lower-level variables. Examples of such scenarios include support vector machines and \(\ell _1\) ℓ 1 and \(\ell _2\) ℓ 2 regularized regression. In this paper, to expand the scope of possible applications, we substantially weaken the key assumption to only require the lower-level program to be convex for any fixed value of the upper-level program. We present an innovative single-level difference of weakly convex reformulation based on the Moreau envelope of the lower-level problem. We further develop a sequentially convergent Inexact Proximal Difference of Weakly Convex Algorithm (iP-DwCA). To evaluate the effectiveness of the proposed iP-DwCA, we conduct numerical experiments focused on tuning hyperparameters for kernel support vector machines on simulated data.