Fractional-Order Activation Functions
摘要
This chapter presents the incorporation of fractional calculus into neural network activation functions to enhance performance. Traditional activation functions like Sigmoid, Tansig, and ReLU are crucial for introducing non-linearity in neural networks, but they have limitations, such as vanishing gradients, non-differentiability, and inactive neurons. Fractional calculus, which generalizes traditional derivatives to non-integer orders, offers a promising approach to overcoming these challenges. By incorporating fractional-order derivatives, this work develops and analyzes fractional-order activation functions, including Binary StepBinary step, Sigmoid, ReLU, and their variants. The book demonstrates that fractional-order activation functions improve adaptability, non-linearity, and predictive accuracy, particularly in complex systems with long-range dependencies.