The deterioration of civil infrastructure, especially bridges, has become a serious societal concern in many countries. Surface damage such as spalling and exposed rebar requires early detection to ensure structural safety and enable timely maintenance. Traditional inspection methods rely heavily on manual visual assessment and annotation, which are labor-intensive and prone to variability. In recent years, object detection models such as You Only Look Once (YOLO) and segmentation models like the Segment Anything Model (SAM) have attracted increasing attention for automating damage detection. In this study, we propose a two-stage method for accurately extracting bridge surface damage by combining YOLOv11 and SAM. YOLOv11 detects candidate damage regions using bounding boxes. These outputs are then used as rectangular prompts for SAM, which generates high-precision pixel-level masks. This hybrid approach leverages YOLO’s fast detection capability and SAM’s segmentation accuracy, achieving both efficiency and detail. The proposed method, which integrates object detection and prompt-based segmentation, provides a practical and accurate solution for bridge damage detection, and demonstrates its potential as a fundamental technology that can contribute to Digital Transformation (DX) of infrastructure maintenance workflows.