Video anomaly detection is a crucial task in surveillance systems, significantly enhancing the safety and security of city residents. It has attracted considerable interest from researchers in computer vision, machine learning, cyber security, remote sensing, and data mining. However, state-of-the-art methods for video anomaly detection still encounter significant challenges in extreme weather conditions such as rain, fog, snow, flood, and thunderstorms. These conditions generate considerable noise and complicate the detection of abnormal events in real-world scenarios, especially in traffic surveillance videos. Therefore, this study investigates the impact of varying weather conditions on the performance of prominent methods for detecting traffic anomalies in aerial videos. We perform extensive experiments using six well-known methods on the benchmark dataset in aerial video surveillance, namely UIT-ADrone. In addition, we provide an in-depth analysis of the practical challenges posed by adverse conditions, including rain and snow. This analysis aims to shed light on the complex scene context that can hinder the effectiveness of current methods in high-altitude drone videos for traffic surveillance.