This chapter presents the incorporation of fractional calculus into neural network activation functions to enhance performance. Traditional activation functions like Sigmoid, Tansig, and ReLU are crucial for introducing non-linearity in neural networks, but they have limitations, such as vanishing gradients, non-differentiability, and inactive neurons. Fractional calculus, which generalizes traditional derivatives to non-integer orders, offers a promising approach to overcoming these challenges. By incorporating fractional-order derivatives, this work develops and analyzes fractional-order activation functions, including Binary StepBinary step, Sigmoid, ReLU, and their variants. The book demonstrates that fractional-order activation functions improve adaptability, non-linearity, and predictive accuracy, particularly in complex systems with long-range dependencies.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Fractional-Order Activation Functions

  • Kishore Bingi,
  • Ramadevi Bhukya,
  • Venkata Ramana Kasi

摘要

This chapter presents the incorporation of fractional calculus into neural network activation functions to enhance performance. Traditional activation functions like Sigmoid, Tansig, and ReLU are crucial for introducing non-linearity in neural networks, but they have limitations, such as vanishing gradients, non-differentiability, and inactive neurons. Fractional calculus, which generalizes traditional derivatives to non-integer orders, offers a promising approach to overcoming these challenges. By incorporating fractional-order derivatives, this work develops and analyzes fractional-order activation functions, including Binary StepBinary step, Sigmoid, ReLU, and their variants. The book demonstrates that fractional-order activation functions improve adaptability, non-linearity, and predictive accuracy, particularly in complex systems with long-range dependencies.