SegFormer-based boundary-aware polyp segmentation with adaptive multi-branch fusion
摘要
Accurate definition of colorectal polyps for the prevention of cancer is challenging due to high variability in appearance and indefinite boundaries. In this work, we present a SegFormer-aided framework comprising a transformer encoder (SegFormer-B4) and a multi-branch fusion head in low-, high-, and all-level scales. Both tasks (edges and segmentation) are predicted by each branch and task-specific, learnable softmax weights then fuse branch logits. The framework is optimized under a composite objective comprising (i) deeply supervised soft dice similarity coefficient (DSC) loss for region overlap, (ii) Lovász–Hinge loss on the fusion head for compatibility for intersection over union (IoU) loss, and (iii) an edge-aware binary cross-entropy (BCE) term supervised by a Laplacian-based thin contour map. A lightweight pre-processing step removes textual overlays (OCR-inpainting-guided and specular highlights attenuation) for the removal of spurious cues. On three public datasets, the approach reaches excellent region and boundary precision: on Kvasir-Seg, mDice=0.946 and mIoU=0.899; on CVC-ClinicDB, mDice=0.961 and mIoU=0.926; and for ETIS (small low-contrast lesions), mDice=0.799 and mIoU=0.705, accompanied by top boundary-sensitive measures (S-measure, weighted