Accurate and timely soil monitoring is crucial for improving agricultural productivity, supporting precision farming, and ensuring food security. To replace traditional laboratory-based time-consuming, and labor-intensive soil analysis method, this study presents a comprehensive framework for real-time soil monitoring that integrates image-based soil classification and spectral data-driven soil nutrient prediction using deep learning techniques to deploy recommended system. Soil classification is performed using RGB images captured by camera mounted on UAV and analyzed through pre-trained ConvNext deep learning model with a training accuracy of 99.74% and testing accuracy of 99%. For Nutrient prediction, a Regression neural network model is used which operates on UV-VIS-NIR spectral band data. This model employs polynomial regression at the neuron level and is optimized using a custom ensemble loss function. Feature selection is enhanced using a Dual Simplex optimization method.