Coreset selection aims to identify small, informative subsets of data that retain the essential characteristics of the full dataset, enabling efficient model training. However, the quality of these subsets is typically evaluated solely based on downstream performance metrics such as accuracy, leaving open questions about what makes a good subset beyond predictive performance. In this paper, we revisit two data quality measures used for evaluating synthetic data (coverage and density) and study how they can be applied to evaluate coreset selection methods. Coverage measures how well the coreset spans the full dataset, while density captures how concentrated the subset samples are within regions densely populated by the full dataset. We evaluate four coreset selection strategies (Uniform sampling, Entropy-based selection, Contextual Diversity, and Graph Cut) on the CIFAR-10 dataset using a ResNet-18 model. Our results show that coverage and density offer valuable insights into the behavior of coreset selection methods, explain performance differences, and highlight trade-offs between exploration and exploitation. Our findings suggest that incorporating coverage and density into the evaluation of coreset selection can inform the design of more effective coreset algorithms and serve as complementary benchmarks for the community to assess and compare dataset quality beyond accuracy alone.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evaluating Coreset Selection with Coverage and Density: A Data Quality Perspective

  • Bárbara Capelo,
  • Maria Russo,
  • André Carreiro,
  • Hugo Gamboa,
  • Duarte Folgado

摘要

Coreset selection aims to identify small, informative subsets of data that retain the essential characteristics of the full dataset, enabling efficient model training. However, the quality of these subsets is typically evaluated solely based on downstream performance metrics such as accuracy, leaving open questions about what makes a good subset beyond predictive performance. In this paper, we revisit two data quality measures used for evaluating synthetic data (coverage and density) and study how they can be applied to evaluate coreset selection methods. Coverage measures how well the coreset spans the full dataset, while density captures how concentrated the subset samples are within regions densely populated by the full dataset. We evaluate four coreset selection strategies (Uniform sampling, Entropy-based selection, Contextual Diversity, and Graph Cut) on the CIFAR-10 dataset using a ResNet-18 model. Our results show that coverage and density offer valuable insights into the behavior of coreset selection methods, explain performance differences, and highlight trade-offs between exploration and exploitation. Our findings suggest that incorporating coverage and density into the evaluation of coreset selection can inform the design of more effective coreset algorithms and serve as complementary benchmarks for the community to assess and compare dataset quality beyond accuracy alone.