Dynamic shortcut connections of deep residual neural network
摘要
Shortcut connections are a well-established technique for constructing very deep convolutional neural networks (DCNNs). These connections directly transfer the output features of one layer to the input of subsequent layers. Most existing approaches employ a static configuration of shortcut connections, which can yield substantial performance improvements for specific tasks. However, this static design may not generalize well across different tasks. It is often infeasible to determine, without extensive trial and error, which residual architecture with predefined shortcut connections will perform best for a given task. Training DCNNs with millions or even billions of parameters already imposes significant computational demands, and the dependency on static connections and task-specific tuning further exacerbates this issue. To address this challenge, we propose an adaptive framework that automatically identifies the most effective set of shortcut connections from all possible candidates for a given task. The proposed method initializes the network with all potential shortcut connections and progressively prunes those that negatively impact performance during training. Experimental results across multiple architectures demonstrate that the proposed adaptive approach consistently outperforms their original static counterparts. Furthermore, the incorporation of dynamic shortcut connections establishes a new class of adaptive network architectures with improved flexibility and generalization capability.