Abstract <p>Colorectal cancer is the second most common cancer in women and the third most common in men. Polyps are the early indicators of colorectal cancer, and colonoscopy is considered the most reliable method for identifying and localizing these polyps. The manual labelling of polyp assessment is time-consuming and subject to human error. Thus, an automated segmentation method can enhance the accuracy and speed of delineating lesion boundaries. Therefore, the ResSegNet++ model is proposed for the semantic segmentation of polyps in this work. The ResSegNet++ model has two encoder-decoder networks, and it implements a squeeze and excitation block along with skip connections to enhance the representational power of the network. The proposed model performs binary segmentation, and the results are illustrated through a segmented mask and an evaluation metric. The highest value of evaluation metric obtained is 0.9764, 0.6680, and 0.9763 for dice coefficient, mean intersection over union, and accuracy, respectively. The proposed model is validated by comparing it with other available models through evaluation metrics and time complexity. In addition, an ablation study is performed to show the importance of each component of the ResSegNet++ model. Moreover, in this work, two different datasets, namely, Kvasir-SEG and CVC-ClinicDB datasets, are utilized to perform all the experiments.</p> Graphical Abstract <p></p>