Deep learning (DL) has changed the landscape of various fields, but its high computing needs make it hard to use in places with limited resources. To address these limitations, this paper presents a novel structured pruning approach that dynamically identifies and removes nonsignificant filters during training using a variance-based threshold. Unlike conventional pruning methods that permanently eliminate filters, our approach applies soft pruning by temporarily deactivating filters with low activation variance, setting their weights to zero, while preserving the original network structure. We retain filters that contribute to some fixed percentage of the total variance in each layer, pruning the rest. Although the variance percentage is fixed, the pruning threshold varies across layers. Importantly, pruned filters remain in the network but are not updated, as their gradients are masked to zero. Experimental evaluations on CIFAR-10, CIFAR-100, and Tiny ImageNet datasets demonstrate that our method reduces computational cost by approximately 58.09% with minimal impact on accuracy.