<p>Tomatoes play a pivotal role in addressing global food needs, providing essential nutrients and economic benefits. Their widespread accessibility makes them vital in combating malnutrition and enhancing food security. However, diseases in tomato plants significantly threaten worldwide production. While recent advances in deep learning have shown promising results in detecting tomato diseases, these studies predominantly rely on datasets captured in controlled laboratory settings. The performance of these techniques often declines when applied to datasets collected in real field environments, primarily due to the complex and varied backgrounds that complicate disease prediction. This study presents a two-stage model to tackle this issue effectively. The first stage focuses on detecting and isolating tomato leaves from images, removing the complex background using a faster region-based convolutional neural network (FRCNN) augmented with feature pyramid networks (FPN). Enhancements are made by incorporating FPNs into the FRCNN framework. In the second stage, the extracted tomato leaves images undergo disease recognition using a customized conformer-based model. This innovative approach, combining the strengths of FRCNN and conformer, is termed CONF-RCNN. The proposed model is tested on the Fieldplant dataset, which consists of images captured in real field conditions. Our approach achieves an impressive accuracy rate of up to 90% in identifying tomato diseases, significantly outperforming existing state-of-the-art convolutional neural network (CNN) models, which generally achieve around 83%. This model demonstrates a robust solution for recognizing tomato diseases in real-world field environments, enhancing the effectiveness of automated disease management systems in agriculture.</p>