Generative dataset distillation aims at knowledge condensation of complete datasets through generative modeling, preserving key training information. Current more advanced methods mostly use diffusion modeling to generate compact synthetic data with high quality and diversity; however, such methods face many challenges in practical applications. For example, it is difficult to cope with long-tailed distributions by treating different categories equally, and it is difficult to adequately express the hierarchical complexity within categories with a large number of hyponyms. In addition, there is the problem of high resource utilization in the process of high-resolution image generation. To cope with the above problems, this paper proposes Efficient and Dynamic Generative Model for Dataset Distillation (EDGM). EDGM proposes dynamic noise control, dynamic clustering center, and dynamic prototype-image generation strategies by introducing a semantic-image dual complexity assessment mechanism focusing on the complex categories. In addition, EDGM introduces PCA dimensionality reduction and dimensionality enhancement strategies in the dynamic clustering phase, which effectively alleviates the memory bottleneck in high-resolution image processing and improves the computational efficiency of the system. Extensive experiments on four standard datasets verify that EDGM offers a good trade-off between accuracy and efficiency, and its comprehensive performance outperforms existing methods.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

EDGM: Efficient and Dynamic Generative Model for Dataset Distillation

  • Gaoyuan Ma,
  • Yonghui Xu,
  • Wei He,
  • Lizhen Cui

摘要

Generative dataset distillation aims at knowledge condensation of complete datasets through generative modeling, preserving key training information. Current more advanced methods mostly use diffusion modeling to generate compact synthetic data with high quality and diversity; however, such methods face many challenges in practical applications. For example, it is difficult to cope with long-tailed distributions by treating different categories equally, and it is difficult to adequately express the hierarchical complexity within categories with a large number of hyponyms. In addition, there is the problem of high resource utilization in the process of high-resolution image generation. To cope with the above problems, this paper proposes Efficient and Dynamic Generative Model for Dataset Distillation (EDGM). EDGM proposes dynamic noise control, dynamic clustering center, and dynamic prototype-image generation strategies by introducing a semantic-image dual complexity assessment mechanism focusing on the complex categories. In addition, EDGM introduces PCA dimensionality reduction and dimensionality enhancement strategies in the dynamic clustering phase, which effectively alleviates the memory bottleneck in high-resolution image processing and improves the computational efficiency of the system. Extensive experiments on four standard datasets verify that EDGM offers a good trade-off between accuracy and efficiency, and its comprehensive performance outperforms existing methods.