The Manchu archives are historical records formed during the Qing Dynasty in China holding significant value, and word spotting serves as a crucial tool for their utilization and development. Due to the necessity of supporting both image and string queries in Manchu archives and the issue of significant shape differences in words, existing document image word spotting methods are inadequate for addressing the challenges of Manchu archive word spotting. To address this, this paper proposes a Manchu archives word spotting method that supports both image and string query modes, based on a multi-task object detection framework. The method defines three sub-networks following the backbone network: pixel classification, bounding box regression, and word embedding. The pixel classification and bounding box regression sub-networks achieve word detection, while the word embedding network is supervised by the embedding vector of the Latin transliteration, achieving the alignment between the image and string representations of Manchu words. Furthermore, by incorporating a multi-scale residual structure and CBAM dual attention mechanism into the backbone network, the method effectively resolves the issue of misdetection caused by significant variations in word shapes in Manchu archives. Finally, we construct the first Manchu archive word spotting dataset, comprising 2,000 Manchu archive images. Experimental results demonstrate that the proposed method not only supports both image and string queries but also significantly outperforms existing methods.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

TMAWS: A Manchu Archives Word Spotting Method Supporting Both Image and String Query Modes

  • Jianjun He,
  • Ligen Cheng,
  • Zihang Zhang,
  • Yu Zhou,
  • Xinshu Cui,
  • Ruirui Zheng

摘要

The Manchu archives are historical records formed during the Qing Dynasty in China holding significant value, and word spotting serves as a crucial tool for their utilization and development. Due to the necessity of supporting both image and string queries in Manchu archives and the issue of significant shape differences in words, existing document image word spotting methods are inadequate for addressing the challenges of Manchu archive word spotting. To address this, this paper proposes a Manchu archives word spotting method that supports both image and string query modes, based on a multi-task object detection framework. The method defines three sub-networks following the backbone network: pixel classification, bounding box regression, and word embedding. The pixel classification and bounding box regression sub-networks achieve word detection, while the word embedding network is supervised by the embedding vector of the Latin transliteration, achieving the alignment between the image and string representations of Manchu words. Furthermore, by incorporating a multi-scale residual structure and CBAM dual attention mechanism into the backbone network, the method effectively resolves the issue of misdetection caused by significant variations in word shapes in Manchu archives. Finally, we construct the first Manchu archive word spotting dataset, comprising 2,000 Manchu archive images. Experimental results demonstrate that the proposed method not only supports both image and string queries but also significantly outperforms existing methods.