The success of CLIP-style pre-training has inspired recent scene text pre-training methods to leverage language supervision for a comprehensive text detector using fully annotated synthetic datasets. However, these approaches predominantly emphasize the semantic meaning of scene text, often neglecting the complex glyph structures that are crucial for accurate text detection. Therefore, this paper introduces Scene Text Reconstructor (STR), a pre-training framework based on Masked Image Modeling (MIM). Our strategy emphasizes text-specific regions to learn finer-grained glyph structures, effectively addressing common challenges like occlusion in scene text detection. STR employs a specialized mask generator that selectively targets text areas, enabling precise reconstruction of masked pixel values through a hierarchical encoder-decoder architecture. By aligning pre-training objectives with text’s structural and contextual properties, STR enhances text feature representation for downstream detection tasks. Experiments across benchmarks demonstrate STR’s superiority, achieving state-of-the-art accuracy and robustness in scene text detection.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Scene Text Reconstructor: A Contextual-Aware Masking Framework for Pre-training Text Detectors

  • Yaohou Fan,
  • Tomo Miyazaki,
  • Zhengmi Tang,
  • Juan Wang,
  • Yongsong Huang,
  • Shinichiro Omachi

摘要

The success of CLIP-style pre-training has inspired recent scene text pre-training methods to leverage language supervision for a comprehensive text detector using fully annotated synthetic datasets. However, these approaches predominantly emphasize the semantic meaning of scene text, often neglecting the complex glyph structures that are crucial for accurate text detection. Therefore, this paper introduces Scene Text Reconstructor (STR), a pre-training framework based on Masked Image Modeling (MIM). Our strategy emphasizes text-specific regions to learn finer-grained glyph structures, effectively addressing common challenges like occlusion in scene text detection. STR employs a specialized mask generator that selectively targets text areas, enabling precise reconstruction of masked pixel values through a hierarchical encoder-decoder architecture. By aligning pre-training objectives with text’s structural and contextual properties, STR enhances text feature representation for downstream detection tasks. Experiments across benchmarks demonstrate STR’s superiority, achieving state-of-the-art accuracy and robustness in scene text detection.