Scholarlink

<p>Image Aesthetic Captioning (IAC) generates text descriptions that reflect an image’s aesthetic qualities. Recent advances in multimodal large models have improved this task but often require significant computational resources. The challenge of efficiently generating diverse aesthetic captions with multimodal large models, while minimizing computational overhead, remains unresolved. To address this gap, we propose a novel prompt-based framework called AesBLIP2 for image aesthetic captioning. AesBLIP2 is composed of three key components: (1) caption filtering and aesthetic attribute-based caption classification, (2) an automatic aesthetic attribute-based prompt generator, and (3) the multimodal large model BLIP2. In our framework, captions are first filtered and classified based on aesthetic attributes. These classified captions are then used to train different templates, generating diverse aesthetic attribute prompt embeddings. During inference, our model selects the appropriate aesthetic attribute prompts to guide BLIP2 in generating descriptions that accurately reflect the aesthetic qualities of the image. Extensive experiments demonstrate that AesBLIP2 is both efficient and effective, requiring minimal computational resources to adapt large multimodal models from the general domain to the aesthetic domain. Furthermore, the generated aesthetic captions are richer and more diverse, underscoring the potential of our approach. Code for implementing AesBLIP2 is available at <a href="https://gitee.com/yongzhenke/aesblip2">https://gitee.com/yongzhenke/aesblip2</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Aesblip2: generating image aesthetic caption via prompting

  • 期刊论文

  • 发表日期：

    2025年8月21日

查看全文
  • Guanjun Sheng,
  • Yongzhen Ke,
  • Shuai Yang,
  • Kai Wang

摘要

Image Aesthetic Captioning (IAC) generates text descriptions that reflect an image’s aesthetic qualities. Recent advances in multimodal large models have improved this task but often require significant computational resources. The challenge of efficiently generating diverse aesthetic captions with multimodal large models, while minimizing computational overhead, remains unresolved. To address this gap, we propose a novel prompt-based framework called AesBLIP2 for image aesthetic captioning. AesBLIP2 is composed of three key components: (1) caption filtering and aesthetic attribute-based caption classification, (2) an automatic aesthetic attribute-based prompt generator, and (3) the multimodal large model BLIP2. In our framework, captions are first filtered and classified based on aesthetic attributes. These classified captions are then used to train different templates, generating diverse aesthetic attribute prompt embeddings. During inference, our model selects the appropriate aesthetic attribute prompts to guide BLIP2 in generating descriptions that accurately reflect the aesthetic qualities of the image. Extensive experiments demonstrate that AesBLIP2 is both efficient and effective, requiring minimal computational resources to adapt large multimodal models from the general domain to the aesthetic domain. Furthermore, the generated aesthetic captions are richer and more diverse, underscoring the potential of our approach. Code for implementing AesBLIP2 is available at https://gitee.com/yongzhenke/aesblip2.