Scholarlink

<p>Fine-grained sketch-based image retrieval (FG-SBIR) aims to retrieve instance-level photo matches from abstract freehand sketches, yet remains challenging due to severe structural abstraction and cross-modal discrepancies between sketches and photographs. While recent CNN- and transformer-based methods improve cross-modal alignment through metric learning or attention mechanisms, they do not explicitly regularize the latent representation against structured perturbations, often leading to unstable embeddings under extreme abstraction. To address this limitation, we propose DiffReg-SBIR, a diffusion-regularized FG-SBIR framework that introduces controlled latent-space noise perturbation to enhance representation stability. Following a variance-preserving diffusion formulation, sketches and photos are projected into a shared latent space, progressively perturbed via a predefined diffusion schedule, and refined through a lightweight U-Net denoiser using single-step noise prediction. A reconstruction-consistency constraint preserves fine-grained identity cues, while joint optimization with metric learning enforces discriminative cross-modal alignment. Extensive experiments on Sketchy, TU-Berlin, and QMUL-Shoe-V2 demonstrate that DiffReg-SBIR consistently outperforms strong CNN-, transformer-, and metric-learning baselines, achieving 60.1% Top-1 accuracy and 70.3% mAP on QMUL-Shoe-V2 and improving mAP by 2.0–3.0% on category-level benchmarks. These results indicate that diffusion-based latent regularization provides an effective and principled mechanism for enhancing embedding robustness in FG-SBIR.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Diffusion-based regularization for fine-grained sketch–photo retrieval under domain and semantic gaps

  • 期刊论文

  • 发表日期：

    2026年6月29日

查看全文
  • Mohammed A. S. Al-mohamadi,
  • C. J. Prabhakar

摘要

Fine-grained sketch-based image retrieval (FG-SBIR) aims to retrieve instance-level photo matches from abstract freehand sketches, yet remains challenging due to severe structural abstraction and cross-modal discrepancies between sketches and photographs. While recent CNN- and transformer-based methods improve cross-modal alignment through metric learning or attention mechanisms, they do not explicitly regularize the latent representation against structured perturbations, often leading to unstable embeddings under extreme abstraction. To address this limitation, we propose DiffReg-SBIR, a diffusion-regularized FG-SBIR framework that introduces controlled latent-space noise perturbation to enhance representation stability. Following a variance-preserving diffusion formulation, sketches and photos are projected into a shared latent space, progressively perturbed via a predefined diffusion schedule, and refined through a lightweight U-Net denoiser using single-step noise prediction. A reconstruction-consistency constraint preserves fine-grained identity cues, while joint optimization with metric learning enforces discriminative cross-modal alignment. Extensive experiments on Sketchy, TU-Berlin, and QMUL-Shoe-V2 demonstrate that DiffReg-SBIR consistently outperforms strong CNN-, transformer-, and metric-learning baselines, achieving 60.1% Top-1 accuracy and 70.3% mAP on QMUL-Shoe-V2 and improving mAP by 2.0–3.0% on category-level benchmarks. These results indicate that diffusion-based latent regularization provides an effective and principled mechanism for enhancing embedding robustness in FG-SBIR.