This paper uses deep learning related technologies to preprocess the pest and disease characteristics of three food crops, rice, corn and wheat, using three different methods to construct their respective pest and disease CNN models for image recognition of a total of 14 pests and diseases of the three food crops. This paper adopts the color segmentation method based on HSV to segment the rice image to be detected, and then divides the training set and the test set, constructs the rice pest and disease model to identify rice pests and diseases. The segmentation results of the preprocessing of the corn image to be detected based on the adaptive threshold and the maximum between-class variance method (OTUS) are compared. Finally, the former is selected to preprocess the corn pests and diseases, and the corn pest and disease CNN model is constructed. For the task of wheat pest and disease identification, this paper adopts image segmentation based on KMeans algorithm. The above three methods are used to identify pests and diseases of three grain crops, and the prediction accuracy rates are 84.7%, 88% and 91.6% respectively.