KAN-PPO: A Fast Convergence and Stable Proximal Policy Optimization Powered by Kolmogorov–Arnold Network
摘要
Proximal Policy Optimization(PPO) is a widely utilized Deep Reinforcement Learning algorithm in fields like robotics, manufacturing, and automated guided vehicles. However, models trained by the ordinary PPO continue to encounter challenges related to model instability and ineffective training, stemming from insufficient learning facilitated by neural networks. Recently, the Kolmogorov–Arnold Network (KAN) was proposed as a function approximation alternative to Multi-Layer Perceptrons (MLPs) because of the high accuracy of learning brought by fine-grained network structure. This paper has integrated KAN into the PPO framework, namely KAN-PPO. Through meticulously designed experiments conducted across OpenAI Gym environments, the KAN-PPO algorithm has demonstrated rapid convergence, stable training processes, and accelerated model learning with the same parameter volume and network layers as the original MLP-PPO. Furthermore, KAN-PPO exhibits effective learning, requiring lighter parameter volumes and network depth when tackling equivalent tasks. Consequently, the proposed KANPPO holds significant potential for application in addressing industrial challenges.