Scholarlink

<p>Penalized linear regression operators that include an <i>L</i><sub>1</sub> penalty, such as the least absolute shrinkage and selection operator (Lasso), have become an important tool in statistical data analysis. Although the <i>L</i><sub>1</sub> penalty makes their objective function convex, it is not differentiable everywhere, motivating the development of proximal gradient algorithms. In this article, we introduce a unified framework to compute closed-form smooth surrogates of a whole class of <i>L</i><sub>1</sub>-penalized regression operators using Nesterov smoothing in a fixed-parameter setting (the problem size <i>n</i> and number of parameters <i>p</i> are fixed). Our contributions are: (1) Our smooth surrogates preserve the convexity of the original (unsmoothed) objective functions, are uniformly close to them, and have closed-form derivatives everywhere for efficient minimization via gradient descent; (2) We prove that the estimates obtained with the smooth surrogates can be made arbitrarily close to the ones of the original (unsmoothed) objective functions and provide explicitly computable a priori error bounds on the accuracy of our estimates; (3) We propose an iterative algorithm to progressively smooth the <i>L</i><sub>1</sub> penalty that increases accuracy and is virtually free of tuning parameters. Since the resulting estimates are typically dense, we explore the use of thresholding to enforce sparsity. We compare our framework to the current gold standards on simulated and real data, demonstrating that our smooth surrogates provide equal or higher accuracy than the gold standards while keeping the aforementioned theoretical guarantees and having roughly the same asymptotic runtime scaling.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A framework to efficiently smooth L1 penalties for linear regression

  • 期刊论文

  • 发表日期：

    2026年2月25日

查看全文
  • Georg Hahn,
  • Sharon M. Lutz,
  • Nilanjana Laha,
  • Julian Hecker,
  • Dmitry Prokopenko,
  • Sanghun Lee,
  • Woojoo Lee,
  • Michael H. Cho,
  • Edwin K. Silverman,
  • Christoph Lange

摘要

Penalized linear regression operators that include an L1 penalty, such as the least absolute shrinkage and selection operator (Lasso), have become an important tool in statistical data analysis. Although the L1 penalty makes their objective function convex, it is not differentiable everywhere, motivating the development of proximal gradient algorithms. In this article, we introduce a unified framework to compute closed-form smooth surrogates of a whole class of L1-penalized regression operators using Nesterov smoothing in a fixed-parameter setting (the problem size n and number of parameters p are fixed). Our contributions are: (1) Our smooth surrogates preserve the convexity of the original (unsmoothed) objective functions, are uniformly close to them, and have closed-form derivatives everywhere for efficient minimization via gradient descent; (2) We prove that the estimates obtained with the smooth surrogates can be made arbitrarily close to the ones of the original (unsmoothed) objective functions and provide explicitly computable a priori error bounds on the accuracy of our estimates; (3) We propose an iterative algorithm to progressively smooth the L1 penalty that increases accuracy and is virtually free of tuning parameters. Since the resulting estimates are typically dense, we explore the use of thresholding to enforce sparsity. We compare our framework to the current gold standards on simulated and real data, demonstrating that our smooth surrogates provide equal or higher accuracy than the gold standards while keeping the aforementioned theoretical guarantees and having roughly the same asymptotic runtime scaling.