Rescue robots use Visual Simultaneous Localization and Mapping (SLAM) for self-positioning and navigation in unfamiliar environments. The presence of dynamic targets and potential dynamic targets in disaster scenarios significantly affects the localization process of visual SLAM, and traditional algorithms cannot eliminate potential dynamic targets. To address these issues, this paper introduces FD-SLAM (FastestDet-SLAM). The algorithm incorporates a new semantic segmentation network and then designs a method that tightly couples semantic and geometric constraints to remove potential dynamic targets, effectively reducing the impact on localization, and utilizing other static feature points to complete the algorithm's localization. Experimental results on five sequences from the TUM dataset show that FD-SLAM achieves a 98.41% improvement compared to ORB-SLAM3. Furthermore, it also outperforms other SLAM algorithms in dynamic scenarios. This algorithm not only ensures real-time processing but also significantly enhances the algorithm's positioning accuracy while maintaining robustness.