<p>Sigma-Pi-Sigma neural network (SPSNN) has gained significant attention due to its exceptional modeling capabilities and superior nonlinear mapping performance. Despite these advantages, the training process of SPSNN poses some challenges. Conventional gradient-based optimization methods often exhibit limitations such as slow convergence and computational inefficiency when applied to SPSNN, negatively impacting the convergence and stability of the model. Therefore, in this paper, a Polak-Ribière-Polyak (PRP) conjugate gradient-based training algorithm is implemented in SPSNN. In contrast to traditional gradient-based training methods, conjugate gradient methods offer advantages in convergence speed and computational efficiency, which significantly enhances model performance. We assume four specific conditions and based on these, results on strong and weak convergence are established. Specifically, the error function presents a monotonic decrease and gradient approaching zero, indicating weak convergence. The weight sequence converges to a fixed point, indicating strong convergence. Detailed proofs of these results are provided. Finally, numerical simulations are conducted across three distinct categories: function approximation problems, parity problems and real-world classification problems. The experimental results not only validate the theoretical findings but also demonstrate the effectiveness of the proposed method. Comparative results also reveal that the proposed method has better performance than other typical methods.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A novel method for Sigma-Pi-Sigma neural network based on Polak-Ribière-Polyak conjugate gradient and its convergence analysis

  • Yan Liu,
  • Zeyong Wu,
  • Jian Li,
  • Jinru Cui,
  • Pengfei Jiang

摘要

Sigma-Pi-Sigma neural network (SPSNN) has gained significant attention due to its exceptional modeling capabilities and superior nonlinear mapping performance. Despite these advantages, the training process of SPSNN poses some challenges. Conventional gradient-based optimization methods often exhibit limitations such as slow convergence and computational inefficiency when applied to SPSNN, negatively impacting the convergence and stability of the model. Therefore, in this paper, a Polak-Ribière-Polyak (PRP) conjugate gradient-based training algorithm is implemented in SPSNN. In contrast to traditional gradient-based training methods, conjugate gradient methods offer advantages in convergence speed and computational efficiency, which significantly enhances model performance. We assume four specific conditions and based on these, results on strong and weak convergence are established. Specifically, the error function presents a monotonic decrease and gradient approaching zero, indicating weak convergence. The weight sequence converges to a fixed point, indicating strong convergence. Detailed proofs of these results are provided. Finally, numerical simulations are conducted across three distinct categories: function approximation problems, parity problems and real-world classification problems. The experimental results not only validate the theoretical findings but also demonstrate the effectiveness of the proposed method. Comparative results also reveal that the proposed method has better performance than other typical methods.