This paper introduces a comprehensive cashew nut detection, classification, tracking, and counting system. YOLOv8 is used to detect and classify cashew nuts into four quality categories. “good”, “peel”, “burn”, and “broken”. The movement of cashew nuts across frames is accurately monitored for tracking tasks by the ByteTrack algorithm. A novel product counting algorithm is proposed for real-time counting of cashew nuts as they pass through the system. The entire system is integrated into a web application using the Streamlit library, offering a user-friendly interface that enables real-time monitoring and configuration. The experimental results demonstrate that the model achieves high detection accuracy, with a precision of 97.4%, recall of 98.7%, mAP@50 of 98.8%, and mAP@50-95 of 82.7%. These results highlight the system’s effectiveness in industrial applications, enhancing production efficiency and ensuring product consistency.