GWOCS: an adaptive hybrid framework for duplicate data removal in cloud systems
摘要
The exponential growth of cloud-hosted data has intensified challenges of redundancy, storage overhead, and degraded efficiency. Traditional deduplication methods, though effective for exact matches, struggle with scalability and near-duplicate detection in distributed environments. This paper presents GWOCS, a hybrid optimization framework that integrates the Grey Wolf Optimizer (GWO) for rapid convergence with Cuckoo Search (CS) for global exploration. A cosine similarity–based fitness function enhances near-duplicate detection, while an adaptive boundary validation mechanism ensures solution stability in high-dimensional search spaces. GWOCS is evaluated on diverse real-world datasets from e-commerce (Amazon Products), healthcare (Indian Liver Records), and IoT (temperature readings) to demonstrate robustness across heterogeneous domains. Experimental results show that GWOCS achieves an average accuracy of 94%, outperforming standalone heuristics (GWO, CS) and conventional baselines such as Genetic Algorithms, Probabilistic Linkage, and Distributed Hash Tables. An ablation study further validates the contribution of hybridization and boundary validation in improving precision and recall. By balancing exploration and exploitation, GWOCS establishes itself as a scalable, accurate, and reliable solution for duplicate data removal in distributed cloud systems, directly contributing to reduced storage costs, improved bandwidth efficiency, and enhanced data quality.