Agriculture is one of the world's most critical strategic sectors, playing a vital role in food security and economic stability. In developing countries like India, crops such as mango and sweet orange hold significant economic and cultural value. However, plant diseases pose a major global challenge, reducing agricultural productivity and potentially leading to food shortages. Early and accurate disease detection is crucial to mitigating these risks and ensuring sustainable farming practices. Traditional disease detection methods are often time-consuming, costly, and prone to errors, highlighting the need for automated solutions. This research proposes a deep learning-based approach to detecting mango and sweet orange leaf diseases, specifically anthracnose and scab, using Convolutional Neural Networks (CNNs) with ResNet-50. This model processes a dataset of leaf images from Kaggle by resizing, noise removal, and feature extraction. The CNN model classifies leaves as healthy or unhealthy, followed by K-means clustering for segmentation and further disease identification. Experimental results demonstrate high classification accuracy, with values ranging from 97.862 to 98.483% across different leaf samples. Sensitivity scores indicate effective disease detection, while mean square error (MSE) and peak signal-to-noise ratio (PSNR) confirm robust image processing. Leaf-2 (mango) was diagnosed with anthracnose, while Leaf-4 (sweet orange) was identified with scab. The results confirm that deep learning provides a reliable and scalable solution for plant disease detection, assisting farmers in improving crop management and reducing yield losses.