Scholarlink

<p>Despite the remarkable success of low-rank estimation in data mining, its effectiveness diminishes when applied to sparse non-negative data that inherently lacks low-rank structure. To address this limitation, in this paper, we focus on non-negative sparse matrices and aim to investigate the intrinsic low-rank characteristics of the rectified linear unit (ReLU) activation function. We first propose a novel nonlinear matrix decomposition framework incorporating a comprehensive regularization term designed to simultaneously promote useful structures in clustering and compression tasks, such as low-rankness, sparsity, and non-negativity in the resulting factors. This formulation presents significant computational challenges due to its multi-block structure, non-convexity, non-smoothness, and the absence of global gradient Lipschitz continuity. We address these issues by developing an accelerated alternating partial Bregman proximal gradient method (AAPB), whose distinctive feature lies in its capability to enable simultaneous updates of multiple variables. Under mild and theoretically justified assumptions, we establish both sublinear and global convergence properties of the proposed algorithm. Through careful selection of kernel generating distances tailored to various regularization terms, we derive corresponding closed-form solutions while maintaining the <i>L</i>-smooth adaptable property. Numerical experiments on graph regularized clustering and sparse NMF basis compression confirm the effectiveness of our model and algorithm.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

An Accelerated Alternating Partial Bregman Algorithm for ReLU-based Matrix Decomposition

  • 期刊论文

  • 发表日期：

    2025年9月27日

查看全文
  • Qingsong Wang,
  • Yunfei Qu,
  • Chunfeng Cui,
  • Deren Han

摘要

Despite the remarkable success of low-rank estimation in data mining, its effectiveness diminishes when applied to sparse non-negative data that inherently lacks low-rank structure. To address this limitation, in this paper, we focus on non-negative sparse matrices and aim to investigate the intrinsic low-rank characteristics of the rectified linear unit (ReLU) activation function. We first propose a novel nonlinear matrix decomposition framework incorporating a comprehensive regularization term designed to simultaneously promote useful structures in clustering and compression tasks, such as low-rankness, sparsity, and non-negativity in the resulting factors. This formulation presents significant computational challenges due to its multi-block structure, non-convexity, non-smoothness, and the absence of global gradient Lipschitz continuity. We address these issues by developing an accelerated alternating partial Bregman proximal gradient method (AAPB), whose distinctive feature lies in its capability to enable simultaneous updates of multiple variables. Under mild and theoretically justified assumptions, we establish both sublinear and global convergence properties of the proposed algorithm. Through careful selection of kernel generating distances tailored to various regularization terms, we derive corresponding closed-form solutions while maintaining the L-smooth adaptable property. Numerical experiments on graph regularized clustering and sparse NMF basis compression confirm the effectiveness of our model and algorithm.