This paper proposes FF-YOLO, a cross-scale feature fusion network guided by attention mechanisms, to address challenges in forest smoke and fire detection—including leakage, false detection, and low accuracy caused by environmental complexity, object occlusion, and insufficient lighting. First, we enhance the YOLOv5s backbone by integrating a Mamba-based linear attention module (MAA). Leveraging Mamba’s efficiency with high-resolution visual tasks, this structurally similar module strengthens effective information extraction, enabling the model to prioritize critical features and improve detection under complex environments and occlusion. Second, we introduce a cross-scale weighted feature fusion structure (NEWFPN) in the neck network. This enhances multi-scale and multi-depth information utilization, overcoming obstacles like low brightness and dense obstructions to boost detection accuracy. Finally, a dedicated small object detection layer is added to the head. Evaluated on our custom dataset, FF-YOLO achieves an 89.25% mAP in detecting forest smoke and fires across diverse conditions—including small fire sources and obstructions—demonstrating its robustness and practical value.