System logs are indispensable for maintaining the reliability and security of modern IT infrastructures, yet they generate vast volumes of data in which anomalies—ranging from performance degradations to security breaches—can be easily overlooked. This review article offers a focused survey of practical statistical methods for log anomaly detection, examining both univariate and multivariate approaches. We begin with classic thresholding techniques such as Z-score and interquartile range (IQR), illustrating their implementation in tools like Prometheus, Zabbix, Splunk, and ELK Stack. To address non-stationarity and seasonal trends, we discuss adaptive strategies based on sliding windows and exponentially weighted moving averages (EWMA), as found in Grafana and Datadog. We then explore multivariate analysis via Mahalanobis distance, highlighting its use in SIEM platforms (IBM QRadar, LogRhythm) and AIOps solutions (Moogsoft). Each method is complemented by real-world use cases and historical context, tracing origins from Shewhart control charts and Tukey’s boxplot to contemporary DevOps practices. By emphasizing transparency, ease of integration, and limitations, this article equips practitioners with a robust first-line toolkit for automated anomaly monitoring. The surveyed statistical techniques serve as the foundation for more advanced machine-learning and hybrid detection systems, which will be addressed in subsequent sections.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Modern Methods for Anomaly Detection in Enterprise System Logs: Algorithms, Implementations, and Practical Case Studies

  • Sergey Yarushev,
  • Aleksandr Anurov

摘要

System logs are indispensable for maintaining the reliability and security of modern IT infrastructures, yet they generate vast volumes of data in which anomalies—ranging from performance degradations to security breaches—can be easily overlooked. This review article offers a focused survey of practical statistical methods for log anomaly detection, examining both univariate and multivariate approaches. We begin with classic thresholding techniques such as Z-score and interquartile range (IQR), illustrating their implementation in tools like Prometheus, Zabbix, Splunk, and ELK Stack. To address non-stationarity and seasonal trends, we discuss adaptive strategies based on sliding windows and exponentially weighted moving averages (EWMA), as found in Grafana and Datadog. We then explore multivariate analysis via Mahalanobis distance, highlighting its use in SIEM platforms (IBM QRadar, LogRhythm) and AIOps solutions (Moogsoft). Each method is complemented by real-world use cases and historical context, tracing origins from Shewhart control charts and Tukey’s boxplot to contemporary DevOps practices. By emphasizing transparency, ease of integration, and limitations, this article equips practitioners with a robust first-line toolkit for automated anomaly monitoring. The surveyed statistical techniques serve as the foundation for more advanced machine-learning and hybrid detection systems, which will be addressed in subsequent sections.