As one of the key technologies for ADAS, traffic sign detection and recognition can effectively obtain traffic sign information on the road such as warnings, instructions, and prohibitions, effectively improving driving safety while ensuring road safety. However, the characteristic expression of traffic signs changes in foggy and night scenes, making the task more challenging in complex scenes. In this paper, we propose a multi-source traffic sign dataset for complex situations, named ITT100K. The construction of this dataset effectively improved the generalization of the model. A SE-YOLOX specific integration model with attention mechanism is proposed. By using the attention mechanism to allocate weights to the output of three complex scenes, the model can focus on feature learning of specific scenes and improve the recognition accuracy of the model in complex scenes. Experiments on ITT100K and foggy driving dataset show that the proposed method has achieved significant improvement.