<p>Self-supervised visual representation learning has shown promising results in recent years, thanks to contrastive learning. However, the problem of false-negative pairing remains a challenge to be addressed. Besides, most self-supervised learning methods are pre-trained on object-centric datasets at the image-level. Yet, pre-training on scene-centric datasets is not explored adequately to learn more comprehensive pixel-level representations for dense-level prediction tasks. Our objective, in this paper, is to alleviate the false-negative pairing problem and to explore pre-training on a scene-centric dataset. For this objective, we introduce three strategies at the pixel-level and pre-train them on COCO, a scene-centric dataset. Firstly, we regularize contrastive learning with consistency loss. Secondly, we combine two matching strategies, based on similarity and distance, used in contrastive learning at the pixel-level. Third, we introduce a relational framework instead of contrastive learning at the pixel-level along with the image-level. The experimental results show that all proposed models outperform the baseline models, namely DenseCL, PixContrast, and ReSSL on COCO object detection and instance segmentation while only the first one surpasses the baselines on the Pascal VOC object detection task with 0.2% AP50 improvement. Moreover, attention map visualization and k-nearest neighbor retrieval results support quantitative results for our contrastive learning-based proposed models.</p>