FusePolyp: Complementary Dual Encoder Design and Fusion Strategy Analysis for Medical Polyp Segmentation
摘要
Accurate segmentation of colorectal polyps is critical for the early detection and prevention of colorectal cancer, yet remains challenging due to significant variability in polyp shape, size, texture, and boundary ambiguity. To address these challenges, we propose FusePolyp, a novel Complementary Dual Encoder Framework that integrates the global semantic modeling ability of transformers with the spatial precision of convolutional features. The architecture leverages a pretrained PVTv2-B3 encoder to capture rich hierarchical semantic representations, complemented by a custom ReSEVM encoder that focuses on local detail through Residual connections, Squeeze-and-Excitation recalibration, and lightweight Visual State-Space Mamba blocks. To effectively combine features from both branches, we introduce the Cross-Mamba Channel-Spatial Fusion (CMCSF) module, which dynamically modulates channel and spatial dependencies while modeling long-range interactions. The fused features are further enhanced through a dual-branch bottleneck consisting of a lightweight transformer and an ASPP module, while a gated decoder with skip connections enables precise boundary reconstruction. Extensive experiments conducted on five benchmark datasets–Kvasir-SEG, CVC-ClinicDB, BKAI-IGH, ETIS, and ColonDB–demonstrate that FusePolyp consistently outperforms state-of-the-art methods, achieving the highest mIoU, Dice, and F \(_2\) scores across all datasets. Notably, FusePolyp delivers improvements of up to 2.0–3.5% in Dice, 2.0–3.8% in mIoU, and 1.5–3.8% in F \(_2\) compared to the best competing models, highlighting its superior segmentation accuracy, robustness, and generalization capability across diverse colonoscopic imaging conditions.