High semantic diversity and intricate boundary details remain significant challenges in remote sensing image segmentation. Existing models often exhibit insufficient inter-class discriminative capability when handling similar land covers due to the lack of robust semantic guidance mechanisms, while their processing of boundary details also demonstrates inadequate precision. To address these limitations, we propose a novel remote sensing image segmentation network named SCSegNet (SAM And CLIP Based Remote Sensing Image Segmentation Network) integrating SAM (Segment Anything Model) and CLIP (Contrastive Language-Image Pretraining). By introducing a Semantic-guided Prompt Enhancement Module, an Adaptive Category Weight Predictor, and a Boundary Feature Extraction Module, the proposed framework dynamically adjusts prompt embeddings using semantic embeddings generated from the CLIP pretrained model. This approach combines adaptive category weights with static weight augmentation to enhance segmentation performance for rare categories. To evaluate the effectiveness of SCSegNet in processing similar land covers, a series of comparative experiments and ablation studies on the ISPRS Potsdam dataset are conducted. Experimental results demonstrate that SCSegNet significantly enhances segmentation accuracy for analogous land cover categories.