Fine-grained object detection refers to the identification and classification of objects in images in a more detailed way in the field of computer vision. Traditional object detection usually recognizes the whole object, while fi-ne-grained object recognition pays more attention to the internal details of the object to distinguish different subclasses in the same category. Fine-grained object detection for remote sensing images faces great challenges due to high line-of-sight, small objects, and less training data. In this work, we design an architecture for fine-grained object detection of remote sensing images of ships at sea. We randomly exchange the ship subject and back-ground in the image through the image mask, and then use the method of image generation to process the dataset augmentation in a unified style, which solves the problem of less fine-grained training data. At the same time, a contrastive learning module is added to the fine-grained image classification subtask, which improves the recognition ability for small target objects in complex backgrounds and the discrimination ability of similar individuals between classes. Our architecture achieves a mAP of 0.749 on the HRSC2016.