In this study, we introduce MS-NET-v2, an enhanced version of the Modular Selective Network (MS-NET), designed to optimize subset construction and expert training. We introduce a cut-off variable, \({\mathcal {O}}\) , to systematically constrain binary class pair sampling based on the Inter-Class Correlation (ICC) matrix. A subset merging algorithm then constructs multi-class subsets encoding coarse concepts. To leverage the non-convexity of deep neural networks, experts are trained from scratch on these subsets, thereby enhancing diversity through the coverage of distinct local minima, leading to improved overall accuracy. Extensive experiments on the CIFAR-10, CIFAR-100, Tiny-ImageNet, and Pets datasets demonstrate the superiority of MS-NET-v2 in accuracy, expert diversity, and parameter efficiency. MS-NET-v2 consistently outperforms heavier ensemble methods in both single and multi-expert settings. For instance, on CIFAR-100, using ResNet-20 as the backbone, MS-NET-v2 achieves 75.10% accuracy with 1.35M active parameters in single-expert settings, surpassing Optimized MS-NET (72.81%) and approaching a five-network ensemble (75.61%, 5.10M parameters). In multi-expert settings, it outperforms the ensemble (77.28% vs. 75.61%). On Tiny-ImageNet, MS-NET-v2 remains competitive in single-expert configurations (59.18% vs. 59.72%) and significantly excels in multi-expert setups (62.63% vs. 59.72%). These results underscore the advantages of MS-NET-v2 over its predecessor and ensemble methods.
