Road damage detection is a very important part in the field of intelligent driving. The early detection method is to embed a large number of sensors in the car for detection. In recent years, deep learning methods have been gradually used in the research of road damage detection. Urban road detection system is a system that uses advanced technology to provide effective support for the safety and maintenance of urban roads. This paper analyzes the functional requirements of the urban road fault detection system, which should have the functions of login and registration, real-time detection, road damage reporting, road damage information viewing, and upload record management. It is mainly divided into three parts: vehicle side, server side and web front end. The experimental results show that the improved algorithm and the urban road damage detection system designed and implemented in this paper have good results and good stability in the actual detection.