A simple yet effective enhanced collaborative filtering framework for mitigating noise and data sparsity: evidence from pervasive digital platform datasets
摘要
In collaborative filtering, numerous studies concentrate on the development of novel similarity measures to enhance predictive accuracy. However, through our extensive case studies, we have identified that these approaches frequently fail to adequately cope with the challenges posed by intrinsic noise and data sparsity, which can significantly impact the reliability of similarity calculations, leading to inaccurate predictions. To address these issues, we propose a straightforward yet effective similarity-enhanced collaborative filtering framework. Initially, we account for item popularity bias when adjusting co-rated items between users and subsequently eliminate rare co-rated items based on a predefined noise threshold. This approach mitigates the influence of unreliable co-rated items (i.e., noise) on similarity results, thereby enhancing the reliability of similarity calculations. Following this step, we employ a simple and effective matrix factorization technique that can be substituted with alternative deep learning approaches to accurately fit and fill the sparse similarity matrix after noise reduction. This strategy alleviates the effects of insufficient nearest neighbors and excessive deviations in similarity due to data sparsity, ensuring high-quality neighborhoods in rating predictions. Experimental results on three public datasets with varying sizes and sparsity levels demonstrate that our framework significantly improves the performance of existing methods using similarity measure. Specifically, compared with traditional similarity methods, our approach achieves notable improvements with average maximum gains of 38.4%, 16.6%, and 18.5% in MAE, HR, and NDCG, respectively. This methodology effectively balances the trade-off between noise reduction and data density while providing a comprehensive solution for optimizing similarity calculations and improving collaborative filtering performance.