Agricultural equipment, such as plant protection unmanned vehicles and breeding robots, face significant challenges in navigating orchard environments characterized by complex terrain, occluded localization signals, and varying meteorological conditions. This paper proposes a visual navigation perception and path planning method for orchards, integrating an enhanced Fast-SCNN model with the Artificial Potential Field (APF) method. A spatial attention mechanism is incorporated to improve the Fast-SCNN model’s performance, after which the APF is applied to compute the optimal navigation path. Experimental results demonstrate that the enhanced Fast-SCNN model achieves a Mean Intersection over Union (mIoU) of 80.2%, important category intersection over union (iIoU) of 93.31%, average pixel accuracy (aAcc) of 96.73%, and mean category pixel accuracy (mAcc) of 85.40% on the test set. The method exhibits strong segmentation capabilities in real-time predictions within the orchard environment and effectively generates reasonable navigation paths that avoid pedestrians based on semantic segmentation outputs. These results indicate that the proposed method provides valuable insights for the visual navigation of orchard plant protection and breeding unmanned vehicles.