iAug: Accelerating Augmentation with Importance Sampling in Deep Neural Network Training
摘要
Data augmentation is a widely used technique in deep learning to enhance model accuracy. It often becomes a performance bottleneck of deep neural network (DNN) training, as they are CPU-intensive. In this paper, we propose iAug, an importance-informed augmentation framework to reduce sample augmentation time in DNN training by selectively applying different numbers of augmentation layers to data samples based on sample importance. First, iAug uses the loss distribution of samples to classify data samples during training to maximize its performance potential. Second, it monitors performance loss due to importance-aware augmentation and uses the error compensation algorithm to adjust augmentation strategies for achieving the targeted accuracy accepted by users. Third, it opportunistically promotes low-importance samples to high-importance samples to improve data diversity and model accuracy. Experiments on standard datasets and DNN models show that iAug reduces preprocessing time by up to 26.2% and total training time by up to 14%, while maintaining or improving accuracy over state-of-the-art methods, proving its effectiveness for efficient DNN training.