With the rapid advancement of deepfake techniques, this technology has attracted significant public attention. Traditional deepfake detection methods primarily focus on distinguishing between real and fake categories, often neglecting the sample level feature differences that lead to these distinctions. As a result, the feature representation distance between each fake sample and its corresponding original real sample is difficult to expand. This prevents the model from capturing discriminative features, ultimately limiting its performance. To address these limitations, we propose a novel deepfake detection framework, Sample Based Contrastive Learning for DeepFake Detection (SCL). SCL leverages the unique characteristics of the deepfake detection task by employing sample based contrastive learning to maximize the feature representation distance between real and fake sample pairs, thereby enhancing classification performance. Specifically, leveraging the temporal correlation of video frames, different frames from the anchor’s video are treated as positive pairs, encouraging them to be drawn closer to the anchor. In contrast, based on the characteristics of the deepfake detection task, video frames of fake or real samples that correspond to the anchor are treated as negative pairs, pushing them further away from the anchor. Experimental results on multiple deepfake detection datasets demonstrate the superior performance of the proposed method. The detection performance on several deepfake detection datasets have demonstrated the performance of SCL.