Segmentation of regions of interest (ROIs) from medical images is considered as a fundamental requirement for many imaging-based clinical decision support systems. Deep learning based automatic segmentation methods have emerged as the state-of-the-art. However, their performance is heavily reliant on using a large number of annotated training dataset to encompass all the possible variations of ROIs, and when the coverage of these variations is inadequate, these methods have difficulties in segmenting the ROIs. Semi-automatic segmentation methods, which fuse user-inputs with high-level semantic image features derived from convolutional neural networks (CNNs) or vision transformers (ViTs) offer an alternative to overcome the limitations of automatic segmentation methods. Unfortunately, CNNs are restricted by the limited receptive fields that cannot capture long-range dependencies, while ViT usually rely on the self-attention mechanism to capture the global context and therefore have high computational complexity. In this study, we propose a Semi-Mamba method for semi-automatic medical image segmentation. The novelty we introduce is to leverage the capability of Mamba to efficiently capture long-range dependency and combine this together with user-clicks to segment challenging ROIs, e.g., ROIs with fuzzy boundaries and inhomogeneous textures. Our experiments with three well benchmarked medical image datasets across different modalities showed that our method consistently outperformed existing automatic and semi-automatic segmentation, which demonstrates strong generalizability.