In the domain of plant disease detection, the adoption of Explainable Artificial Intelligence (XAI) is imperative to shed light on the decision-making processes of models. This study explores the significance of XAI in enhancing the interpretability of models designed for plant disease detection. Additionally, Federated Learning, specifically employing the FedAvg algorithm, is implemented to address data privacy concerns and improve prediction accuracy. With a dataset comprising 1530 samples, a centralized custom Convolutional Neural Network (CNN) model forms the basis of the implementation. The integration of Federated Learning demonstrates a notable accuracy enhancement, elevating the model’s overall performance from 91.3 to 96.6%. To build confidence in these models, XAI methodologies are introduced, providing transparent explanations for predictions. This comprehensive approach not only advances the field of AI in agriculture but also ensures trustworthiness, privacy, and improved performance in plant disease detection systems.