The granular computing can mimic the way humans think when dealing with large-scale complex problems and has been effectively utilized in the area of anomaly detection. However, most anomaly detection algorithms based on granular computing are incapable of managing incomplete data. Throughout this paper, we propose the fuzzy information entropy tailored for incomplete data, along with its application in anomaly detection algorithms for incomplete data. First, fuzzy set and fuzzy similarity relation with respect to incomplete data are defined. Second, the fuzzy entropy and its related metrics are proposed, which include incomplete fuzzy information entropy, incomplete fuzzy joint entropy, incomplete fuzzy conditional entropy, incomplete fuzzy mutual information, and incomplete fuzzy complement entropy. Additionally, two indicators, the incomplete entropy ratio factor and incomplete potential difference, are proposed. Subsequently, an anomaly detection model is established using the proposed theoretical approach. Initially, a relation matrix is formed using the fuzzy similarity relations of the incomplete information system. Next, the incomplete entropy ratio factor and incomplete potential difference are calculated for specific attribute sets of attributes. Then, anomaly factors for objects within specific attribute sets are formulated, and anomaly scores are calculated to evaluate the extent of anomaly in these objects. Finally, the related anomaly detection algorithm, ADIFIE (Anomaly Detection for Incomplete data based on Fuzzy Information Entropy), is proposed. Comparative experiments with mainstream anomaly detection algorithms are carried out on public UCI datasets and synthetic datasets. Experimental results demonstrate that the ADIFIE algorithm attains the best performance in the conducted experiments, excelling in three-quarters of the datasets.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Anomaly Detection Using Fuzzy Information Entropy for Incomplete Data

  • Yuhao Tang,
  • Chang Liu,
  • Zhong Yuan,
  • Chun Gong,
  • Ning Feng,
  • Shihao Wang

摘要

The granular computing can mimic the way humans think when dealing with large-scale complex problems and has been effectively utilized in the area of anomaly detection. However, most anomaly detection algorithms based on granular computing are incapable of managing incomplete data. Throughout this paper, we propose the fuzzy information entropy tailored for incomplete data, along with its application in anomaly detection algorithms for incomplete data. First, fuzzy set and fuzzy similarity relation with respect to incomplete data are defined. Second, the fuzzy entropy and its related metrics are proposed, which include incomplete fuzzy information entropy, incomplete fuzzy joint entropy, incomplete fuzzy conditional entropy, incomplete fuzzy mutual information, and incomplete fuzzy complement entropy. Additionally, two indicators, the incomplete entropy ratio factor and incomplete potential difference, are proposed. Subsequently, an anomaly detection model is established using the proposed theoretical approach. Initially, a relation matrix is formed using the fuzzy similarity relations of the incomplete information system. Next, the incomplete entropy ratio factor and incomplete potential difference are calculated for specific attribute sets of attributes. Then, anomaly factors for objects within specific attribute sets are formulated, and anomaly scores are calculated to evaluate the extent of anomaly in these objects. Finally, the related anomaly detection algorithm, ADIFIE (Anomaly Detection for Incomplete data based on Fuzzy Information Entropy), is proposed. Comparative experiments with mainstream anomaly detection algorithms are carried out on public UCI datasets and synthetic datasets. Experimental results demonstrate that the ADIFIE algorithm attains the best performance in the conducted experiments, excelling in three-quarters of the datasets.