Data imbalance is one of the most important issues in the process of training deep learning networks. It directly affects the performance of the proposed models. Most recent methods have focused on improving the model’s performance by proposing new rebalance strategies, using more networks, transfer learning, etc. In particular, Balanced Meta-Softmax (BLMS) is a simple method that focuses on adjusting the softmax function to better fit imbalanced data via pre-calculating the number of samples of each parameter. However, this adjustment is constant throughout training. This seems inappropriate when applying data augmentation. In this paper, we proposed a simple yet robust approach to overcome the above drawback by calculating the number of samples that incorrectly predicted the label at epoch \(t-1\) to adjust the balance softmax function at epoch t. The experiment results on various long-tailed datasets have shown that the proposed framework significantly outperforms different methods.