With the advantages of safety, reliability and low cost, trams have gradually attracted people's attention and developed into one of the main rail transit modes for small and medium-sized traffic. Since trams mostly adopt the form of semi-independent right of way, in this mixed operation mode, trams need to share the right of way with vehicles and pedestrians, which leads to obstacles on the tram track and affects the safety of operation. Firstly, this paper studies the image preprocessing, including image graying, image denoising, image binarization, image edge detection and other methods. The appropriate method is selected to process the obstacle image, and then the detection window is established on the basis of the track. The processed obstacle image is detected on the detection window, and variance and black-and-white pixel ratio is used to determine whether there are obstacles on the track, which plays a certain reference role for the safety of trams operation.