This paper constructs a virtual simulation dataset with multiple complex environments for End-to-End autonomous vehicles, considering the significant impact of complex environments on severity of accidents. First, statistics are conducted using from real autonomous vehicles accident reports, DMV and NHTSA, to analyze the correlation between lighting, weather, and tire slippage with accident severity. Second, within the Carla simulation platform, 200 km of On-Board camera images and LiDAR point cloud data are gathered from eight towns using a Rule-Based autonomous vehicle model. This collection forms a “Long-Tail” scenario dataset, which comprises eight complex environment combinations and covers diverse road types, including intersections, roundabouts, highways, and villages. It also includes scenarios like following, lane changing, and avoidance. Finally, the experiments validate the Transfuser and Interfuser End-to-End autonomous vehicles models using the collected datasets. The impact of the complex environment dataset is analyzed by comparing metrics such as route completion, infraction score, and driving score. Compared to the normal environment, the results show that the driving scores of the two End-to-End models in the complex environment decrease by 33.2% and 13.1% averagely. This demonstrates the effectiveness of the constructed dataset. The dataset and data collection code are available at https://github.com/littleblackzi/WeatherCore .