A Survey of Overfitting Mitigation Methods in Deep Learning: From Regularization to Data Augmentation
摘要
Deep learning algorithms, a key subset of artificial intelligence, are increasingly employed across diverse domains to handle complex data. In fact, they outperform traditional methods. These networks rely on a fundamental building block—the artificial neuron—which computes a weighted sum of its inputs, applies an activation function (e.g., sigmoid, tanh, ReLU, or Leaky ReLU), and propagates the output. However, a major challenge in supervised learning with deep neural networks is overfitting, where the model memorizes training data instead of generalizing to unseen examples. This paper critically examines the most widely used techniques to mitigate overfitting in deep learning. We focus on the most prominent approaches: (1) data augmentation, which artificially expands the training dataset; (2) regularization (L1/L2), which penalizes excessive weights to reduce model complexity; (3) dropout, a randomization technique that prevents co-adaptation of neurons during training; and early stopping. For each technique, we present a conceptual explanation and discuss typical use cases. The paper does not include new experimental results; a comparative summary table is provided to guide practitioners in selecting appropriate strategies for reducing overfitting, particularly in resource-constrained scenarios.