Content-orthogonal projection for disentangled style transfer in diffusion models
摘要
Modern diffusion-based style transfer relies on cross-attention to fuse content structures with reference styles. However, conventional cross-attention often entangles structural cues with textural patterns from the style image, leading to shape leakage and distortion of the source content. To address this issue, we propose a Content-Orthogonal Projection mechanism. This mechanism explicitly removes the subspace of style values aligned with content representations by projecting style features onto the orthogonal complement of the content subspace. This design ensures that cross-attention conveys only texture-, color-, and brush-related statistics while strongly suppressing structural artifacts. We integrate this method into the reweighted cross-attention framework, enabling zero-shot style transfer. Extensive experiments demonstrate that our method faithfully preserves content geometry while significantly enhancing stylistic fidelity, achieving superior disentanglement between style and structure. This work provides an efficient and principled pathway toward controllable, texture-oriented style transfer in diffusion models.