In this study, we demonstrate an all-in-one grading system that integrates a robotic pick-and-place system with fruit quality grading system. The system consists of a conveyor belt which gently moves fruits along its path, while a 5-DOF robotic arm delicately picks up each fruit and positions it in front of a camera. Using advanced computer vision models such as YOLO, MobileNet SSD, and Faster R-CNN, the system determines whether a fruit is fresh or rotten with an accuracy of over 91% within 10–12 s. Fruits are either categorized as good or bad based on a threshold. Fresh fruits are then directed into good baskets, while the rest of the fruits are removed with the help of a conveyor belt to the bad basket. ROS ensures synchronization of all components which results in a cost-effective and precise system. During training, we used techniques like color jittering and random erasing to enhance the model’s ability to recognize diverse visual patterns. Recall and mAP@50–95 are used to assess the effectiveness of our system, which shows a reduction in large labor cost and time commitment needed.