​‍​‌‍​Pest​‍​‌‍​‍‌ infestations cause significant losses in crops; thus, farmer incomes get reduced and the use of chemical pesticides escalates. Hence, revealing the pest issue at its earliest stage and providing a timely forecast is the only way to rescue the crop in a safe and ecologically friendly manner. This paper introduces a machine-learning framework that can classify pests by pictures and estimate the spread of the pest using a weather-based forecast. Convolutional Neural Network (CNN) models locate pests in images captured from the field as well as in reference images, and the time-series models take environmental and historical data to compute the probability of an outbreak. This is very much in line with the principles of Integrated Pest Management (IPM) as it implies the use of spraying only in the target areas and not as a frequent or routine pesticide application. The research outcomes have demonstrated that the models tested on both benchmark and semi-field datasets have excellent performance in terms of detection and short-term forecasting and are thus able to achieve better results than the baseline methods. This study shows how detection, prediction, and decision-based recommendations can work together to reduce pesticide use. This reduction supports crop health and leads to stable yields.