Accurate polyp segmentation is vital for diagnosing colorectal cancer but faces challenges due to varying sizes, colors, and clinical conditions. Despite advancements, deep learning systems still have significant limitations in effectively detecting and segmenting polyps. Convolutional Neural Network-based methods struggle to capture long-range semantic relationships, whereas Transformer-based approaches often fail to understand local pixel interactions effectively. Moreover, these methods sometimes inadequately extract detailed features and face limitations in scenarios requiring optimized local and global feature modeling. To tackle these challenges, we introduce DOLG-CNet, a novel one-stage, end-to-end framework specifically crafted for polyp segmentation. Initially, we employ the cutting-edge ConvNeXt for its superior segmentation capabilities. Additionally, we integrate an orthogonal fusion module that adeptly merges global and local features to generate a rich combined feature set. We also introduce a unique training strategy that marries contrastive learning with segmentation training, enhanced by an auxiliary deep supervision loss to boost performance. Specifically, we create both high and low augmented versions for each input image and train the system to align their vector embeddings closely, regardless of the augmentation level. This method, combined with standard segmentation loss and deep supervision, facilitates faster and more effective convergence. Our experimental results demonstrate that DOLG-CNet achieves impressive performance, with a dice coefficient score of 0.913 on Kvasir-SEG, 0.761 on CVC-ColonDB, and 0.722 on ETIS. Additionally, in qualitative and quantitative benchmarks across various datasets, DOLG-CNet consistently outperforms well-known methods, proving its efficacy and potential in the field.