Scholarlink

The Hamiltonian Deep Neural Networks (H-DNNs) is an innovative neural network architecture that integrates the energy conservation and gradient stability principles from Hamiltonian mechanics, effectively addressing the gradient vanishing and interpretability issues commonly encountered in traditional deep learning methods. Firstly, this study empirically validates the effectiveness of H-DNNs in complex tasks, demonstrating its applicability to challenging image recognition problems. However, the H-DNNs employs a fixed time step, which presents limitations when handling complex data and fails to adequately capture critical features during the model training process. To address these issues, a novel architecture, Adaptive Hamiltonian Deep Neural Networks combined with Efficient Channel Attention (AdaHamil-ECA), is proposed in this paper. This architecture introduces an adaptive time-step mechanism that intelligently adjusts the time step based on data complexity and the model’s learning state, ensuring that the model has sufficient time to integrate and propagate information when processing complex features while avoiding unnecessary computational resource usage. Additionally, the embedded ECA-Net module in AdaHamil-ECA learns the importance and interrelationships of different channels, precisely enhancing the weights of critical feature channels while suppressing irrelevant ones. This enables the model to focus more on the feature information that significantly contributes to the task goal, thereby improving discriminative power and generalization performance. Experimental results show that AdaHamil-ECA outperforms traditional H-DNNs by a 2.7%-3% increase in accuracy for complex image classification tasks, while also accelerating convergence and significantly enhancing robustness.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Fusion of ECA-Net with Hamiltonian Deep Neural Networks and Dynamic Step Size Optimization

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Yongqi Liang,
  • Shuangcheng Bai,
  • Zhiyi Zhang

摘要

The Hamiltonian Deep Neural Networks (H-DNNs) is an innovative neural network architecture that integrates the energy conservation and gradient stability principles from Hamiltonian mechanics, effectively addressing the gradient vanishing and interpretability issues commonly encountered in traditional deep learning methods. Firstly, this study empirically validates the effectiveness of H-DNNs in complex tasks, demonstrating its applicability to challenging image recognition problems. However, the H-DNNs employs a fixed time step, which presents limitations when handling complex data and fails to adequately capture critical features during the model training process. To address these issues, a novel architecture, Adaptive Hamiltonian Deep Neural Networks combined with Efficient Channel Attention (AdaHamil-ECA), is proposed in this paper. This architecture introduces an adaptive time-step mechanism that intelligently adjusts the time step based on data complexity and the model’s learning state, ensuring that the model has sufficient time to integrate and propagate information when processing complex features while avoiding unnecessary computational resource usage. Additionally, the embedded ECA-Net module in AdaHamil-ECA learns the importance and interrelationships of different channels, precisely enhancing the weights of critical feature channels while suppressing irrelevant ones. This enables the model to focus more on the feature information that significantly contributes to the task goal, thereby improving discriminative power and generalization performance. Experimental results show that AdaHamil-ECA outperforms traditional H-DNNs by a 2.7%-3% increase in accuracy for complex image classification tasks, while also accelerating convergence and significantly enhancing robustness.