<p>Recently, pre-trained vision-language models like CLIP have demonstrated significant zero-shot inference ability in visual recognition tasks. However, current test-time prompting alignment methods still face challenges in adapting CLIP to the region-level visual relation detection (VRD) task, due to CLIP’s lack of region-level awareness and its single-prototype representation assumption. An intuitive baseline is to input the union region of the object pair and the hand-crafted text prompt into CLIP for zero-shot VRD, which suffers from two inherent drawbacks: (1) The cropped visual inputs often lose focus when other objects are included, (2) a single text prompt struggles to match the diverse visual appearances of the relation. In this paper, we propose a fine-grained multimodal prompt learning method to precisely adapt CLIP for VRD in a decoupled, staged execution paradigm. First, we use the Segment Anything Model to design the fine-grained visual prompt, known as the blurred reverse mask, which is combined with the object pair’s union region to create composite visual prompts. Second, we introduce diverse, learnable text prompts, which are optimized through multivariate Gaussian distribution modeling to capture the intra-class diversity of visual relations. Finally, we adopt Llama 2 to validate the rationality of relations for further improvement. Extensive experiments are conducted on two challenging VRD datasets, demonstrating that our method outperforms state-of-the-art methods, enhancing the performance of CLIP in VRD. Furthermore, qualitative results also show that our method generalizes well to novel, unseen relation categories.</p>