Introduction Fine-tuning pre-trained vision models has shown promise in addressing downstream tasks in biomedical imaging. However, traditional segmentation methods rely heavily on manual annotations, which are time-consuming and labor-intensive. Deep learning models often require extensive labeled datasets, a challenge in the biomedical domain due to the scarcity of annotated data. Additionally, the domain shift between natural and medical images limits the applicability of pre-trained models like the Segment Anything Model (SAM). This study explores the fine-tuning of the Segment Anything Model (SAM) to overcome these limitations, aiming for zero-shot generalization and flexible prompting for segmentation using medical images. SAM, a vision foundation model that has been pre-trained on a massive dataset of images, is adapted to a specific biomedical imaging task from a pre-trained state, leveraging its learned representations. Methodology The study employs a downstream dataset comprising electron microscope sections from the CA1 hippocampus region, each measuring 5x5x5µm. These volumetric images are divided into smaller patches (patch_size = 256, step = 256) for efficient training, resulting in 1987 image-mask pairs of size 256x256. To ensure robust evaluation, the dataset was split into training and validation sets, with 80% of the data used for training and 20% for validation. Cross-validation was not explicitly performed in this study, which is a limitation that will be addressed in future work. The fine-tuned model was also validated using another downstream task on ultrasound breast cancer dataset comprising 130 images with annotated ground truth masks. SAM's architecture involves a vision transformer (ViT-B) for image encoding, a simple text encoder for prompt conversion, and a lightweight transformer for mask decoding. Training employs the Adam optimizer with various loss functions (e.g., Dice Cross-Entropy, Dice Focal, Generalized Dice Loss) and learning rate scheduling over 200 epochs. Results Fine-tuning SAM on the biomedical imaging task yields improved performance compared to its pre-trained state. The model demonstrated an overall Dice score of 0.82 without prompts, which increased to 0.9 when point-based prompts were applied during inference. For loss function optimization, the Generalized Dice Loss outperformed other methods, achieving a final loss value of 0.045 at the 200th epoch, compared to 0.06 for both Dice Cross-Entropy Loss and Dice Focal Loss. These results highlight SAM's adaptability and effectiveness for specific biomedical segmentation tasks. Conclusion The successful fine-tuning of SAM underscores its versatility and applicability in biomedical imaging tasks. The study emphasizes the importance of leveraging pre-trained models and flexible prompting techniques for efficient adaptation to specific domains. Further exploration into SAM's performance under different loss functions and optimization strategies can provide insights into its robustness and generalization capabilities, paving the way for enhanced biomedical image analysis techniques.