This paper addresses the challenge of unsupervised anomaly detection in key metrics related to data transmission in fourth and fifth-generation (4G and 5G) cellular networks, with the main aim of developing a monitoring service that can alert on abnormal behavior in real-time. The dataset used in this research includes channel and upper-layer performance metrics collected from several cellular modems deployed on a remote-controlled ship. Deep autoencoders (DAE) were selected as the main approach due to their ability to detect point, contextual, and collective anomalies. Several unsupervised machine learning algorithms and preprocessing techniques were applied and optimized to overcome the problem of using DAEs for unsupervised anomaly detection, namely their need for training data containing no known anomalies. Different types of deep autoencoders, including classical, variational, denoising, and robust models, as well as different neural network architectures, consisting of linear and recurrent layers, along with the additional loss function terms, such as contractive and sparse regularizations, have been explored. The presented approach is evaluated using anomalies generated by unsupervised algorithms and special samples that introduce noise and shuffling into normal data, thus affecting the data distribution. Finally, a post-analysis is performed by adding contextual information to the data to help interpret the detected anomalies introducing another layer of validation to the presented approach.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Unsupervised Anomaly Detection in Cellular Modem Metrics Using Deep Autoencoders

  • Nikita Smirnov,
  • Mika Friesenborg,
  • Sven Tomforde

摘要

This paper addresses the challenge of unsupervised anomaly detection in key metrics related to data transmission in fourth and fifth-generation (4G and 5G) cellular networks, with the main aim of developing a monitoring service that can alert on abnormal behavior in real-time. The dataset used in this research includes channel and upper-layer performance metrics collected from several cellular modems deployed on a remote-controlled ship. Deep autoencoders (DAE) were selected as the main approach due to their ability to detect point, contextual, and collective anomalies. Several unsupervised machine learning algorithms and preprocessing techniques were applied and optimized to overcome the problem of using DAEs for unsupervised anomaly detection, namely their need for training data containing no known anomalies. Different types of deep autoencoders, including classical, variational, denoising, and robust models, as well as different neural network architectures, consisting of linear and recurrent layers, along with the additional loss function terms, such as contractive and sparse regularizations, have been explored. The presented approach is evaluated using anomalies generated by unsupervised algorithms and special samples that introduce noise and shuffling into normal data, thus affecting the data distribution. Finally, a post-analysis is performed by adding contextual information to the data to help interpret the detected anomalies introducing another layer of validation to the presented approach.