Evaluating Coreset Selection with Coverage and Density: A Data Quality Perspective
摘要
Coreset selection aims to identify small, informative subsets of data that retain the essential characteristics of the full dataset, enabling efficient model training. However, the quality of these subsets is typically evaluated solely based on downstream performance metrics such as accuracy, leaving open questions about what makes a good subset beyond predictive performance. In this paper, we revisit two data quality measures used for evaluating synthetic data (coverage and density) and study how they can be applied to evaluate coreset selection methods. Coverage measures how well the coreset spans the full dataset, while density captures how concentrated the subset samples are within regions densely populated by the full dataset. We evaluate four coreset selection strategies (Uniform sampling, Entropy-based selection, Contextual Diversity, and Graph Cut) on the CIFAR-10 dataset using a ResNet-18 model. Our results show that coverage and density offer valuable insights into the behavior of coreset selection methods, explain performance differences, and highlight trade-offs between exploration and exploitation. Our findings suggest that incorporating coverage and density into the evaluation of coreset selection can inform the design of more effective coreset algorithms and serve as complementary benchmarks for the community to assess and compare dataset quality beyond accuracy alone.