Integrating machine learning algorithms on circuits with low power consumption and low hardware complexity is challenging at the different levels of the design process. In network design at the software level, it is crucial to balance a high classification accuracy, while minimizing model complexity to reduce hardware demands. This paper proposes a search approach integrated with the Neural Architecture Search (NAS) to enhance the performance and reduce the complexity of deep learning models. Accuracy and number of Floating-Point Operations Per Second (FLOPS) are employed as evaluation metrics for the targeted models. The experimental results demonstrate that the proposed method outperforms similar state-of-the-art architectures while exhibiting comparable accuracy with up to a 70% reduction in complexity.