Medical image segmentation becomes increasingly important for identifying and delineating anatomical structures, diseases, and abnormalities within medical images. However, existing large pre-trained foundation models for medical image segmentation, such as MedSAM, are often limited by their substantial computing resource requirements when fine-tuned for specific tasks. This high computational burden hinders their deployment in real-world clinical scenarios where rapid model adaptation is crucial yet computing resources are typically constrained. To address the challenge, in this paper, we introduce LoRA-MedSAM, a highly efficient method for fine-tuning large pre-trained models on medical image segmentation tasks. In particular, we leverage the Low-Rank Adaptation technique to significantly expedite the fine-tuning process of MedSAM model while maintaining high performance. Extensive experiments on three widely adopted medical image segmentation datasets have demonstrated both the effectiveness and efficiency of our method. Specifically, LoRA-MedSAM drastically reduces the fine-tuning time by 22.7%, 14.8% and 29.4% while outperforming the vanilla MedSAM by 1.1%, 5.5% and 0.5% higher Dice Similarity Coefficient (DSC) on the AMOS, BTCV and MultiTalent datasets, respectively.