Scholarlink

<p>Deep Neural Networks have been deployed as the dominant models with outstanding results across a wide range of applications. However, when facing high-dimension and low sample-size data, deep models suffer from overfitting which reduces their generalization performance on new data. To tackle this problem and diminish the gap between recognition rates on training and test sets, weight regularization methods such as L1 and L2 are commonly used. In deep neural networks, the L1 regularization does not consider correlated features, leading to overfitting by reducing the coefficients of the features to zero. On the other hand, the L2 regularization penalizes large weights, which can result in the deep model being less sensitive to features that describe rare data. To address these issues, we propose a novel regularization method using partial derivative pooling that captures more robust deep features. Our regularization method is based on Jacobian matrix calculation to compressed knowledge in all dimensions, resulting in improved representation and generalization performance of DNNs. Experimental results demonstrate that our approach has achieved significant improvements in performance generalization, with gains of 6.63%, 6.11%, and 3.82% accuracy increase on CIFAR-10, CIFAR-100, and ImageNet datasets, respectively. These results highlight the effectiveness of our method in achieving better performance and more robust generalization across a different dataset.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Partial derivative regularized knowledge transformation for deep neural networks generalization

  • 期刊论文

  • 发表日期：

    2025年12月27日

查看全文
  • Sajedeh Morabbi,
  • Hadi Soltanizadeh,
  • Saeed Mozaffari,
  • Mohammad Javad Fadaeieslam

摘要

Deep Neural Networks have been deployed as the dominant models with outstanding results across a wide range of applications. However, when facing high-dimension and low sample-size data, deep models suffer from overfitting which reduces their generalization performance on new data. To tackle this problem and diminish the gap between recognition rates on training and test sets, weight regularization methods such as L1 and L2 are commonly used. In deep neural networks, the L1 regularization does not consider correlated features, leading to overfitting by reducing the coefficients of the features to zero. On the other hand, the L2 regularization penalizes large weights, which can result in the deep model being less sensitive to features that describe rare data. To address these issues, we propose a novel regularization method using partial derivative pooling that captures more robust deep features. Our regularization method is based on Jacobian matrix calculation to compressed knowledge in all dimensions, resulting in improved representation and generalization performance of DNNs. Experimental results demonstrate that our approach has achieved significant improvements in performance generalization, with gains of 6.63%, 6.11%, and 3.82% accuracy increase on CIFAR-10, CIFAR-100, and ImageNet datasets, respectively. These results highlight the effectiveness of our method in achieving better performance and more robust generalization across a different dataset.