A new optimization model for regularization in CNNs using spatial correlation and guided by particle Swarm optimization
摘要
Convolutional neural networks learn discriminative feature structures that are crucial for complex tasks, though their high complexity makes them susceptible to overfitting, particularly when dropout is applied to convolutional layers, where spatially correlated activations limit their effectiveness. Although structured regularization methods such as DropBlock address this issue, they do not take into account semantic information and may drop discriminative regions essential for classification. Thus, a structured form of dropout is needed that considers these important spatial correlations. In this case, we propose a novel regularization method, called PSO-Drop, which introduces a new optimization model for regularization. To solve this optimization problem, we define a new particle structure, where each particle represents a candidate solution in the particle swarm optimization (PSO) framework, explicitly preserving discriminative regions with strong spatial correlations. Extensive experiments demonstrate that PSO-Drop improves classification accuracy and outperforms state-of-the-art regularization methods.