Recently, augmentation-based contrastive learning has made significant progress in avoiding hard backpropagation by enhancing supervision signals to optimize intermediate layers. As a well-known observation, same-source augmentations from the same image are more similar than same-class augmentations from different images but in the same class. However, the existing contrastive deep supervision methods ignore the differences in augmentations between the same-source and same-class images, and then neglect the unique information extraction of the same-source, resulting in a reduced model performance. To tackle this limitation, we design a novel module to independently consider same-source and same-class losses, which assists the neural network in understanding the invariance of same-source augmentations and the commonality of same-class augmentations. Furthermore, the proposed module prevents the effect of same-class losses on same-source losses. Experimental results on several standard datasets with ten models show that our proposed method significantly improves the image classification performance of models in both supervised and semi-supervised learning. Code and models will be released at GitHub .