The utilization of Deep Neural Network (DNN) models is widespread in various commercial applications, including virtual assistants and chatbots, as well as across diverse domains such as manufacturing, medical, autonomous driving, defense, and security. DNN models excel in addressing pattern-based challenges like object identification, image categorization, and annotation. While these models exhibit exceptional accuracy, their high computational, storage, and power requirements pose challenges for deployment on edge devices. To overcome these constraints, our previous work introduced a novel optimizer called “SparPen,” designed explicitly for filter pruning. It significantly surpasses the current state-of-the-art filter pruning technique called Stripe-Wise Pruning (SWP). With a combined approach of our proposed SparPen optimizer and SWP on a VGG-16 model trained on the CIFAR-10 dataset, our results demonstrate an impressive improvement of 73.4% in terms of memory reduction compared to SWP alone. SparPen employs l1 and l2 regularization techniques, and the hyperparameters can be fine-tuned by choosing appropriate alpha and beta values toward optimizing the model. In this work, we propose a meta-learning algorithm to determine the optimal values for alpha and beta. By leveraging these techniques, we aim to fine-tune SparPen to achieve the best possible results to address variance and variety in the dataset. The proposed optimized VGG-16 model exhibited improvement in accuracy by 0.21% for CIFAR-10 and 4.66% for CIFAR-100 and reduced the model size by 97.66% for CIFAR-10 and 92.91% for CIFAR-100 compared to the reported VGG-16 models. Furthermore, our exploration extended work on the ImageNet dataset. In this context, we achieved a reduction in model size by 80.5% with a corresponding accuracy drop of 5%.