In medical image segmentation, manual annotation is an exceptionally costly process, highlighting the critical need for selecting the most valuable samples for labeling. Active learning provides an effective solution for selecting informative samples, however, they face the challenge of cold start, where the initial training samples are randomly chosen, potentially leading to suboptimal model performance. In this study, we present a novel cold start active learning framework based on Segment Anything Model (SAM), which leverages the zero-shot capabilities of SAM on downstream datasets to address the cold start issue effectively. Concretely, we employ a multiple augmentation strategy to estimate the uncertainty map for each case, then calculate patch-level uncertainty corresponding to the patch-level features generated from SAM’s image encoder. Then we propose a Patch-based Global Distinct Representation (PGDR) strategy that integrates patch-level uncertainty and image features into a unified image-level representation. To select the samples with representative and diverse information, we propose a Greedy Selection with Cluster and Uncertainty (GSCU) strategy, which effectively combines the image-level features and uncertainty to prioritize samples for manual annotation. Experiments on prostate and left atrium segmentation datasets demonstrate that our framework outperforms five state-of-the-art methods as well as random selection in various selection ratios. For both datasets, our method achieves comparable performance to that of the fully-supervised method with only 10% and 1.5% annotation burden. Code is available at https://github.com/Hilab-git/SUGFW.git .