Fuzzy Logic Estimation of Probability Value for the Dropout Function Applied to Convolutional Neural Networks
摘要
Within the realm of neural networks, various regularization techniques are used to prevent overfitting. One of the most common methods is the dropout function, which randomly deactivates a fraction of neurons during training. This approach reduces dependency on specific neurons. However, standard dropouts apply a fixed probability of deactivation, which may not always be optimal for different databases. Consequently, this work aims to enhance this function by adapting the dropout rate based on a fuzzy system that estimates the optimal probability according to the network architecture. The values obtained during the training process such as validation loss and validation accuracy are those that allow obtaining the inputs of the fuzzy system to obtain the new probability value for the next training epoch. The present work was implemented in a convolutional neural network architecture replacing the dropout function and testing it using MNIST and Fashion-MNIST databases. This shows that fuzzy logic can be used in the dropout function. The proposed method allowed us to obtain 0.9208 accuracy as the best experiment in the Fashion-MNIST database, surpassing the traditional Dropout.