This paper proposes a custom neural network model based on MobileNetV3_Small, optimized using structured pruning techniques to classify passion fruit flowers efficiently. We combine several data-augmentation methods, including brightness, contrast, saturation, hue adjustments, flipping, and rotation, to enhance the model’s robustness in handling various real-world conditions. The custom model reduces computational complexity and memory footprint by removing the SE module and reducing the number of inverted residual blocks, while maintaining high classification accuracy. Experimental results on the proprietary dataset show that our model achieves a top-1 accuracy of 97.41%, with a significant increase in inference speed and a model size of only 2.84 MB, making it highly suitable for resource-constrained devices such as micro drones. Our approach effectively balances model efficiency and performance, providing a practical solution for real-time applications in precision agriculture. Future work explores further optimizations and broader applications in other fields.