Urban rail transit is an indispensable mode of transportation in today's society, known for its large capacity, low transportation costs, and environmental benefits. However, manual inspection struggles to cover the tens of thousands of kilometers of track, and foreign object intrusion accidents in urban rail can cause casualties and economic losses. With the development of deep learning technology, perception systems can automatically extract key features from data and recognize critical objects in images, providing technical support for preventing foreign object intrusions. Therefore, this paper proposes a method that combines semantic segmentation and object detection to identify intrusive foreign objects by calculating the pixel distance between potential foreign objects and the track area. First, a track segmentation dataset is constructed based on an urban rail test line, and the efficient and reliable DDRNet is introduced to extract the track area, achieving a mIoU of 92.87% and an accuracy of 86.13% for single-track categories. Second, the high-performance Yolov8 is used to capture the location information of potential foreign objects, with an average accuracy of 98.2%. Finally, by determining whether the pixel distance between the track and the foreign object is less than a threshold, intrusive foreign objects are effectively distinguished.