Data augmentation is a widely used technique in deep learning to enhance model accuracy. It often becomes a performance bottleneck of deep neural network (DNN) training, as they are CPU-intensive. In this paper, we propose iAug, an importance-informed augmentation framework to reduce sample augmentation time in DNN training by selectively applying different numbers of augmentation layers to data samples based on sample importance. First, iAug uses the loss distribution of samples to classify data samples during training to maximize its performance potential. Second, it monitors performance loss due to importance-aware augmentation and uses the error compensation algorithm to adjust augmentation strategies for achieving the targeted accuracy accepted by users. Third, it opportunistically promotes low-importance samples to high-importance samples to improve data diversity and model accuracy. Experiments on standard datasets and DNN models show that iAug reduces preprocessing time by up to 26.2% and total training time by up to 14%, while maintaining or improving accuracy over state-of-the-art methods, proving its effectiveness for efficient DNN training.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

iAug: Accelerating Augmentation with Importance Sampling in Deep Neural Network Training

  • Rubayet Rahman Rongon,
  • Xuechen Zhang

摘要

Data augmentation is a widely used technique in deep learning to enhance model accuracy. It often becomes a performance bottleneck of deep neural network (DNN) training, as they are CPU-intensive. In this paper, we propose iAug, an importance-informed augmentation framework to reduce sample augmentation time in DNN training by selectively applying different numbers of augmentation layers to data samples based on sample importance. First, iAug uses the loss distribution of samples to classify data samples during training to maximize its performance potential. Second, it monitors performance loss due to importance-aware augmentation and uses the error compensation algorithm to adjust augmentation strategies for achieving the targeted accuracy accepted by users. Third, it opportunistically promotes low-importance samples to high-importance samples to improve data diversity and model accuracy. Experiments on standard datasets and DNN models show that iAug reduces preprocessing time by up to 26.2% and total training time by up to 14%, while maintaining or improving accuracy over state-of-the-art methods, proving its effectiveness for efficient DNN training.