The primary cause of colorectal cancer, the world's biggest cause of cancer-related death, is colorectal polyps. Late polyp detection during colonoscopy is essential for prompt detection and intervention. Models for deep learning, specifically convolutional neural networks (CNNs) as well as more recently, transformer-based architectures have achieved significant progress in automated segmentation methods for polyp detection. However, current models face significant challenges in defining polyps, particularly because of the large variability in polyp shape, size, texture and color, which frequently blend with neighboring tissues. In this paper, we introduce a new polyp segmentation model using a UNETR (U-Net Transformer) architecture with an improved attention block, specifically tailored for the complexity in polyp segmentation. In contrast to prior work which combines CNNs and transformers at the same layer, we use multi-level transformer encodings with convolutional up-sampling to jointly encode high level semantic information and fine grained details of polyp structure. This multi-level integration allows our model to learn dynamically varying feature contributions according to contextual relevancy and improves segmentation accuracy and robustness to a variety of scenarios. With a Dice Similarity Coefficient (DSC) of 0.87 and an Intersection over Union (IoU) of 0.77 during training, and a Dice score of 0.77 and an IoU of 0.64 during validation, we show that our model performs significantly better in segmentation and far better than traditional CNN-based and transformer-based approaches in terms of accuracy and computational efficiency on benchmark datasets. This makes a significant step forward in the development of real-time, clinically viable polyp segmentation for improved diagnosis and treatment planning.