Accurate segmentation of medical images plays a critical role in clinical diagnosis and patient treatment planning. With the recent advent of powerful Multimodal Large Language Models (MLLMs) and foundational models such as the Segment Anything Model (SAM), various approaches are emerging in the field of medical imaging, aiming to improve segmentation accuracy and generalizability. However, existing SAM models are highly dependent on prompts, requiring extensive manual input, and MLLMs struggle to effectively capture the complex anatomical structures and subtle texture details inherent in medical images, limiting their direct applicability to medical segmentation tasks. Additionally, segmentation models relying solely on images face limitations in freely segmenting target organs or handling multiple organs simultaneously. To address these limitations, we propose a novel text-driven segmentation framework that leverages both textual and visual information to enhance segmentation performance. Our framework integrates a BLIP-based text encoder and a Swin Transformer image encoder through a feature mixer to effectively combine textual information with spatial cues and facilitate inter-modal relationship learning. By leveraging our predefined text codebook, our approach enables more flexible and adaptive segmentation, demonstrating the capability to selectively segment target anatomical regions across multiple organs. Moreover, evaluations on multiple benchmark datasets—specifically, AbdomenCT-1K, and TotalSegmentatorV2—demonstrate that our method consistently outperforms state-of-the-art models in terms of segmentation accuracy and generalization.