Effective detection of human behavioral anomalies remains critical in surveillance systems, yet poses significant challenges due to anomalies’ rarity and their unpredictable nature as unseen events during training. We introduce MADDA, a Multi-scale Adaptive Dynamic approach for skeleton-based Anomaly Detection that addresses the fundamental limitations of existing methods: inadequate temporal modeling and static representation of normal behaviors. MADDA features two innovative components: (1) Multi-scale Temporal-Spatial Feature Pyramid (MTSF) which processes skeletal sequences at multiple time scales simultaneously, capturing both instantaneous and progressive anomalies; and (2) Dynamic Weighted Latent Space Center Update (DWCU) which adaptively adjusts center positions based on distributional characteristics, significantly improving discrimination between marginal normal behaviors and genuine anomalies. Evaluated across three geometric spaces—Euclidean, hyperbolic, and radial—MADDA consistently outperforms state-of-the-art methods on HR-UBnormal (68.7 AUC), HR-ShanghaiTech (77.8 AUC), and HR-Avenue (87.9 AUC) while maintaining computational efficiency. Ablation studies confirm both components substantially contribute to performance, with MTSF showing particular effectiveness in hyperbolic space.