Low-rank adapters (LoRAs) have emerged as an efficient method for customizing large-scale diffusion models, but their internal representations remain poorly understood. We present a comprehensive investigation of the interpretability of adapter weight-spaces for image diffusion models. To that end, we open-source a dataset of 100,000 Stable Diffusion adapters fine-tuned across a hierarchy of image concepts amounting to 264 leaf classes, complete with training metadata. Through systematic analysis, we demonstrate that adapter weights encode meaningful semantic information about their training data, enabling direct interpretation without image generation. We evaluate multiple weight-space representations, including raw parameters, statistical summaries, and learned embeddings, to determine their effectiveness in predicting training data characteristics. To demonstrate real-world impact, we apply our findings to the critical task of detecting potentially harmful content on newly introduced NSFW (Not Safe For Work) toy dataset of Stable Diffusion LoRAs fine-tuned on harmful content. This work advances the interpretability of adapter-based fine-tuning and provides practical tools for understanding and auditing adapted diffusion models.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Towards Weight-Space Interpretation of Low-Rank Adapters for Diffusion Models

  • Jacek Duszenko,
  • Piotr Bielak

摘要

Low-rank adapters (LoRAs) have emerged as an efficient method for customizing large-scale diffusion models, but their internal representations remain poorly understood. We present a comprehensive investigation of the interpretability of adapter weight-spaces for image diffusion models. To that end, we open-source a dataset of 100,000 Stable Diffusion adapters fine-tuned across a hierarchy of image concepts amounting to 264 leaf classes, complete with training metadata. Through systematic analysis, we demonstrate that adapter weights encode meaningful semantic information about their training data, enabling direct interpretation without image generation. We evaluate multiple weight-space representations, including raw parameters, statistical summaries, and learned embeddings, to determine their effectiveness in predicting training data characteristics. To demonstrate real-world impact, we apply our findings to the critical task of detecting potentially harmful content on newly introduced NSFW (Not Safe For Work) toy dataset of Stable Diffusion LoRAs fine-tuned on harmful content. This work advances the interpretability of adapter-based fine-tuning and provides practical tools for understanding and auditing adapted diffusion models.