As the complexity of deep neural networks (DNNs) grows, there is an increasing demand for efficient and lightweight training methods, especially for edge devices with limited resources. As a model compression technique, pruning is a promising solution to improve training efficiency by removing redundant neurons and connections. It typically involves two approaches: train-prune-retain, which incurs additional resource costs, and pruning-while-training, which struggles to balance the model size and accuracy in the one-step manner. In contrast, progressive pruning provides more granular control over the pruning schedule and intensity, facilitating a better balance between model sparsity and accuracy. Inspired by this, we propose a two-stage progressive pruning-based training method. The top-k weights are identified and updated using the learning rate, while the remaining weights are adjusted only through a weight decay mechanism. In the second stage, progressive pruning is applied to the tracked weights, with the untracked weights being pruned at the beginning. During the progressive pruning, a nested model library is generated, which enables efficient switching between different balance points of sparsity and accuracy. When applied to resource-limited edge devices, the library provides greater flexibility to trade off resource cost and model accuracy. This method is applied to train VGG-S, DenseNet, MobileNetV2, and ResNet18, using the CIFAR-10, CIFAR-100, and SVHN datasets. Experimental results show that our method achieves a 20× compression rate with less than 1% accuracy loss compared to baseline models (trained without pruning). Compared to existing pruning-based training methods, our method yields the least accuracy loss and convergence time, while maintaining a similar compression rate.