Recent advancements in handwritten text generation using diffusion models have achieved high-quality and realistic handwriting synthesis. However, existing models often suffer from limited style variability, which reduces their effectiveness for downstream tasks like Handwriting recognition, writer identification, which rely on diverse handwriting samples to ensure model generalization and robustness. Without sufficient variability, models trained on synthetic data risk overfitting to a narrow set of styles, limiting their applicability in real-world scenarios. Diffusion models typically require large-scale datasets to generalize effectively, but in the domain of handwriting generation, comparatively limited training data is available, leading to strong memorization tendencies. As a result, generated handwriting samples often replicate training data instead of introducing novel variations, further restricting their usefulness for downstream applications. To address this, we propose a training-free style mixing approach that dynamically injects multiple writer styles at inference time, enabling controlled handwriting diversity. Our method leverages a pre-trained diffusion model, allowing flexible writer style transitions at the character level without modifying model weights. By introducing controlled style variation, our approach mitigates memorization effects and enhances the diversity of generated samples, making them more suitable for training and evaluation purposes.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Beyond Memorization: Training-Free Style Mixing for Variability in Handwritten Text Generation Using Writer Embedding Injection in Pretrained Diffusion Models

  • Aniket Gurav,
  • Sukalpa Chanda,
  • Narayanan C. Krishnan

摘要

Recent advancements in handwritten text generation using diffusion models have achieved high-quality and realistic handwriting synthesis. However, existing models often suffer from limited style variability, which reduces their effectiveness for downstream tasks like Handwriting recognition, writer identification, which rely on diverse handwriting samples to ensure model generalization and robustness. Without sufficient variability, models trained on synthetic data risk overfitting to a narrow set of styles, limiting their applicability in real-world scenarios. Diffusion models typically require large-scale datasets to generalize effectively, but in the domain of handwriting generation, comparatively limited training data is available, leading to strong memorization tendencies. As a result, generated handwriting samples often replicate training data instead of introducing novel variations, further restricting their usefulness for downstream applications. To address this, we propose a training-free style mixing approach that dynamically injects multiple writer styles at inference time, enabling controlled handwriting diversity. Our method leverages a pre-trained diffusion model, allowing flexible writer style transitions at the character level without modifying model weights. By introducing controlled style variation, our approach mitigates memorization effects and enhances the diversity of generated samples, making them more suitable for training and evaluation purposes.