To address the issues of poor handling of ground object change boundary details in complex scenes and insufficient synergy caused by the decoupling of semantic and change subtasks in existing semantic change detection methods for remote sensing images, this paper proposes an improved algorithm based on contrastive learning. A Cross-Layer Attention Fusion FPN Decoder is designed. Through a multi-stage feature fusion strategy, it integrates low-level spatial details and high-level contextual information to generate more discriminative feature representations. Additionally, a Pixel-Level Change Modeling with Classification Features method is proposed, which shifts change modeling from the decision layer to the feature fusion stage. A dynamic change-aware loss is introduced to establish a collaborative training mechanism between the semantic segmentation and change detection tasks, enhancing the model’s sensitivity to change regions. In contrastive learning, a boundary hard sample mining strategy is proposed to prioritize the optimization of category boundary pixels, strengthening the model’s ability to recognize complex boundaries. The method is validated on two public datasets. Experimental results show that, compared with existing methods, the proposed method achieves significant improvements on multiple evaluation metrics for the tasks of change detection and semantic segmentation.