Accurate segmentation of 3D blood vessels has significant value in clinical diagnosis and surgical planning. However, challenges such as thin tubular structures, low contrast, complex branching patterns, and a scarcity of high-quality annotated data pose substantial difficulties for deep learning-based methods. With the emergence of pre-trained visual-language foundation models such as Contrastive Language-Image Pre-training (CLIP), research on general computer vision tasks has entered a new stage. However, in vessel segmentation tasks, these models still exhibit significant performance shortcomings. To fully leverage the cross-modal semantic capabilities of CLIP, we propose a novel fine-tuning framework called AdaBCA-CLIP, which uses pre-trained CLIP model as a powerful feature extractor for 3D CT scans. Building on this, we introduce a specially designed adapter module paired with a self-adaptive learning strategy to effectively fuse text and image modality embeddings. In addition, we design a Bi-directional Cross-Attention module to facilitate complementary information exchange between image and text features. The experiment was conducted on the public PARSE2022 dataset and our private PAV-Seg3D dataset. Experiments have shown that AdaBCA-CLIP outperforms other state-of-the-art methods in metrics such as Dice, Jaccard, and HD95.