Weeds in the context of agricultural research refer to undesired plant species that compete with cultivated crops for essential nutrients, potentially affecting crop health and yield. Effective weed removal is imperative for ensuring the well-being of the cultivated crop and preserving the integrity of the surrounding ecosystem within the scope of agricultural research. This research underscores the critical importance of accurate weed classification as a foundational element in promoting efficient weed management and control strategies. In this research, we have utilized the particle swarm optimization algorithm’s functionality to evolve the YOLO model’s near-optimal architecture for effective weed classification. To enhance weed classification, we sourced data from two distinct origins, namely ICAR-DWR and TNAU, and applied image segmentation and augmentation techniques during the data preprocessing phase. The findings of this investigation provided empirical validation for the superiority of the particle swarm optimization approach compared to conventional YOLO designs and contemporary state-of-the-art techniques in the domain of weed classification. The proposed approach demonstrated a noteworthy precision of 98.98% during the assessment of the TNAU weed dataset, while also delivering an exceptional 97.89% accuracy on the ICAR-DWR weed dataset. The study’s results have far-reaching consequences for developing environmentally responsible weed control practices that meet all applicable regulations.