Spark-based DBSCAN++ for efficient density-based clustering
摘要
Density-based spatial clustering of applications (DBSCAN) is a widely adopted clustering algorithm known for its ability to discover clusters of arbitrary shape and handle noise. DBSCAN++ mitigates this challenge by evaluating a reduced set of candidate core points through sampling. However, for massive datasets, even DBSCAN++ introduces significant computational complexity, making single-machine implementations inefficient. To address these limitations, distributed computing frameworks such as Apache Spark have been increasingly adopted. By distributing the core-point determination steps and computation of the neighborhood search, Spark-based DBSCAN implementations offer improved scalability and performance, enabling clustering on datasets with millions of points. In this paper, we propose a Spark-based DBSCAN++, which employs an efficient sampling strategy, called Linspace, that ensures even coverage across data distributions. Our comprehensive experiments on synthetic and real-world datasets demonstrate that the proposed Spark-based DBSCAN++ significantly reduces runtime and memory consumption without compromising clustering accuracy compared to other combinations of DBSCAN variants and sampling strategies. The findings underscore the effectiveness of combining distributed processing with an optimized sampling strategy to advance scalable density-based clustering.