The article presents an algorithm supporting the detection of errors in databases. Unlike typical approaches, where correctness rules are defined a priori, in the proposed approach, they are constructed adaptively based on the statistical analysis of historical data. A huge number of automatically generated potential features are first statistically analyzed using the kernel method, and then, similarity indices between features are determined using the Rajski/Jaccard entropy-based metric. The algorithm ensures a balance between the individual usefulness of the features and their independence from others. A low-dimensional feature vector is obtained, which is the input to a classifier based on a neural network. The results of comparative experimental studies are presented, and the usefulness of the algorithm in practical application is discussed.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Entropic Metrics in Feature Selection for Neural Network-Based Data Error Detection in CMDB

  • Grzegorz Mzyk,
  • Szymon Niewiadomski

摘要

The article presents an algorithm supporting the detection of errors in databases. Unlike typical approaches, where correctness rules are defined a priori, in the proposed approach, they are constructed adaptively based on the statistical analysis of historical data. A huge number of automatically generated potential features are first statistically analyzed using the kernel method, and then, similarity indices between features are determined using the Rajski/Jaccard entropy-based metric. The algorithm ensures a balance between the individual usefulness of the features and their independence from others. A low-dimensional feature vector is obtained, which is the input to a classifier based on a neural network. The results of comparative experimental studies are presented, and the usefulness of the algorithm in practical application is discussed.