A two-stage deep learning framework for oil palm fresh fruit bunch ripeness classification toward predictive harvesting in precision agriculture
摘要
Efficient harvest timing and accurate maturity assessment of oil palm fresh fruit bunches (FFBs) are critical for optimizing oil extraction rates, plantation yield, and operational efficiency. Conventional manual inspection is subjective, labour-intensive, and unsuitable for large-scale deployment, while existing automated approaches often fail to capture the multi-stage ripeness progression required for predictive harvest decision-making. This study presents a two-stage deep learning framework for automated FFB ripeness classification and predictive harvesting in precision agriculture. The proposed system integrates You Only Look Once version 8 (YOLOv8)-based pixel-level instance segmentation with a Real-Time Detection Transformer (RT-DETR) classifier to enable robust four-category ripeness grading based on exocarp colour percentage, comprising Unripe (0–20%), Unripe-minor-red (21–40%), Underripe (41–59%), and Ripe (60–100%). This formulation supports immediate harvest decisions and provides additional maturity information that may assist harvest planning and readiness assessment. The framework was developed using 1,200 field-collected red, green, and blue (RGB) images acquired from a commercial plantation in Selangor, Malaysia, which were subsequently augmented to generate a final dataset of 6,000 images. The segmentation stage achieves 98.12% mean average precision at an intersection-over-union threshold of 0.5 (mAP@0.5), effectively isolating FFBs from complex backgrounds through mask-guided preprocessing. The classification stage leverages hybrid convolutional and transformer-based feature learning to capture both local texture and global spatial patterns. Evaluated on a held-out test set, the proposed framework achieves 95.17% accuracy, 95.17% F1-score, and 99.42% receiver operating characteristic area under the curve (ROC–AUC), outperforming six benchmark architectures. The integrated system operates at 55–120 ms per frame (8–18 frames/s), satisfying real-time requirements for autonomous harvesting platforms. These results demonstrate that the proposed framework has strong potential as a scalable and field-deployable solution for precision agriculture, enabling objective ripeness assessment and supporting harvest planning, labour allocation, and mill capacity management in oil palm plantations.