<p>Medical image segmentation is crucial for identifying and diagnosing various diseases, but it remains challenging due to the complexity and diversity of medical imaging data. Traditional approaches often struggle to accurately identify boundaries and capture complex features, particularly in multimodal images. To address these challenges, this research presents ABRB-Net, an advanced segmentation model that combines an Attention-Based Boundary Refinement Block (ABRB) with a Decomposed Residual Convolutional Block (DRB). The key objective of this method is to improve the accuracy of boundary segmentation while improving the feature representation of medical images. The ABRB component is designed to improve boundary areas by focusing attention on significant edge features that are usually overlooked by traditional segmentation methods. Meanwhile, DRB decomposes complex convolutional processes, resulting in more efficient and useful feature extraction. This combined approach allows the model to address the challenges that arise in various medical imaging modalities. The proposed ABRB-Net architecture is evaluated on multiple medical imaging datasets, including brain tumor MRI scans, lung CT scans, lung X-rays, skin cancer dermoscopy images, breast ultrasound scans, polyp colonoscopy images, and a combined dataset. The results show that ABRB-Net surpasses state-of-the-art methods with higher Dice scores. The proposed method significantly enhances boundary delineation and segmentation performance, making it an effective tool for medical diagnosis and treatment planning.</p>