The proposed work aims to implementing panoptic segmentation to classify the maritime scene into foreground and background. Such classification is a major challenge for applications like observing the environmental challenges, maritime control, and autonomous navigation to ensure safe and effective operations at various maritime scenes. Utilizing the R-CNN ResNet-101 and Panoptic FPN model on the COCO dataset, the model is trained using the Detectron2 library for object identification and segmentation. It is intended to particularly parse a variety of object types in complex maritime environments, such as obstacles, boats, and ships. The model’s performance in identifying and classifying maritime items has been demonstrated by the training process, achieving an average accuracy of 72%. The result demonstrates De-tectron2’s adaptability in various environmental conditions, which qualifies it for practical maritime applications. The result highlights the effectiveness of Detectron2 in maritime object detection and segmentation, as well as its adaptability in different environmental conditions.