An outlier detection algorithm based on local density feedback
摘要
Outlier detection is very important in the field of data mining and is applied to various scenarios, such as financial fraud detection and network intrusion. Traditional outlier detection methods usually detect outliers based on the local density of objects, and have achieved some results. However, some challenges still exist: (1) traditional methods only consider neighbor information when calculating the density of an object and ignore the global information embedded in the dataset, leading to the wrong detection of outliers and normal points that are only detected in the global view; (2) traditional methods focus only on comparing the density of an object to its neighbors, ignoring the similarity to its neighbors, leading to incorrectly detecting normal points in sparse regions as outliers, even if all the neighbors of a given point are normal ones. To address these issues, we propose a novel outlier detection algorithm based on the local density feedback (LDF). Our method utilizes principal component analysis (PCA) and a natural neighbor search for initial density estimation. A feedback mechanism is designed to refine the density iteratively by leveraging neighborhood similarity, and to aggregate global information for a more accurate outlierness depiction. By integrating local and global data characteristics, our method reliably detects outliers across diverse datasets. Experimental results on ten datasets show that the LDF algorithm outperforms the existing methods by 10.7% and 1.89% on average in terms of precision and AUC, respectively.