Image classification is one of the most important aspects of computer vision with direct applications in precision agriculture and sustainable development. In the current paper, we introduce a new methodological framework to automatic healthy and infected potato leaves classification, from the perspective of more effective crop management and prevention of unnecessary use of plant protection products. Our approach integrates statistical modelling in the field of wavelets, scale-invariant feature transformation (SIFT) and a convolutional neural network (CNN). The developed model is robust and has very rich representation, and it significantly improves the detection of leaf symptoms. The experimental results confirm that the integration of the SIFT algorithm strengthens the performance of the model, achieving better results compared to existing state-of-the-art approaches on comparable datasets. Apart from its methodological contribution, this research highlights the importance of computer vision as a tool to aid sustainable agriculture through facilitating early disease detection and in attaining sustainable development of food security and rational use of resources.