<p>Medical image analysis, particularly in segmentation, represents a pivotal frontier for deep learning innovation. Its wide-ranging applications, intrinsic complexities, data availability nuances, collaborative intersections, automation prospects, and fertile research trajectories drive it. Segmentation, a cornerstone of medical image analysis, entails delineating precise anatomical structures or regions within diverse imaging modalities such as Ultrasound, MRI, CT scans, and Dermoscopic images. This task is critical for disease diagnosis, treatment strategizing, and patient surveillance in clinical settings. Despite the evolution of segmentation techniques over time to surmount the limitations of conventional methods, a notable paradigm shift is witnessed with the burgeoning ascendancy of transformer-based models eclipsing traditional CNN architectures. This review meticulously investigates the underpinning mechanisms of attention and the intricate architectural nuances of transformers vis-à-vis their application in medical image segmentation. A systematic examination of 88 research papers from prominent academic databases shows a preference for U-Net-based transformer architectures, especially with integrated transformer elements into the encoder framework. Such hybrid models have demonstrated markedly superior segmentation efficacy compared to their conventional counterparts. However, persistent challenges loom, including the lack of meticulously annotated medical data indispensable for robust model training. To address this hurdle, the review deliberates on potential stratagems like transfer learning and harnessing foundational models tailored for specialized medical segmentation tasks. By elucidating prevailing trends, delineating inherent constraints, and charting promising avenues for future inquiry, this review aspires to catalyze the ongoing trajectory of transformer-based methodologies in surmounting clinical difficulties encountered in medical image segmentation.</p>