With the continuous improvement of agricultural automation, addressing the issue of target detection and classification for diversified agricultural products, this study proposes an automated sorting and classification system that combines a Genetic Algorithm (GA) and Random Forest (RF). The GA is utilized to optimize feature selection for agricultural products, while the RF is employed for classification, ultimately achieving precise and efficient sorting operations. The research results demonstrate that the GA-RF algorithm exhibits excellent performance in target detection, with an accuracy of 91%, a recall rate of 90%, and an F1-score of 90%. Compared to traditional algorithms, the GA-RF algorithm significantly reduces sorting times in various agricultural product sorting tasks, achieving times of 14.1 s, 12.8 s, 13.4 s, etc., with misclassification rates of 3.4%, 3.1%, and 3.5%, respectively. Furthermore, the system receives high evaluations across multidimensional ratings from different practitioners, particularly in terms of ease of operation, accuracy, and stability, gaining recognition from most operators and technicians. The GA-RF algorithm not only improves the accuracy of agricultural product sorting and classification but also effectively enhances sorting efficiency, making it particularly suitable for agricultural image sorting tasks under complex backgrounds.