Recent advancements in medical image segmentation have been driven by deep learning’s capability to extract rich features from extensive datasets. However, these improvements rely heavily on large annotated datasets, which pose significant challenges in the resource-intensive medical field. Foundational models, such as Meta’s Segment Anything Model (SAM), have been developed to address these challenges. SAM has demonstrated exceptional zero-shot performance, often rivaling or surpassing fully supervised models across various tasks. Nonetheless, SAM cannot be directly applied to medical image segmentation due to domain shift, making it necessary to fine-tune the model using prompts. Reducing the annotation workload is crucial to alleviate the burden and constraints associated with extensive data annotation in the medical field. This study investigates prompt-guided strategies in SAM for medical image segmentation under few-shot and weakly supervised scenarios. We assess various strategies-bounding boxes, positive points, negative points, and their combinations-using two publicly available datasets. Optimal results are achieved using positive-negative points, demonstrating that the SAM model can perform comparably to established methods in hepatic vascular and prostate cancer segmentation, even with minimal examples. This research aims to advance medical image segmentation by decreasing reliance on extensive annotated data, providing insights into effective prompt utilization, and showcasing SAM’s adaptability in specialized medical contexts.