The Research on End-to-End Tibetan Text Detection and Recognition in Natural Scenes
摘要
Aiming at the low detection and recognition accuracy caused by the lack of natural image data of Tibetan language, the excessive background noise of natural scenes and the similarity of Tibetan characters, this paper utilizes the existing laboratory dataset (NSTD) for detecting and recognising Tibetan language in natural scenes. Meanwhile, an end-to-end Tibetan text detection and recognition method is proposed, which adopts Swin Transformer as the feature extraction module, combines with the Transformer Encoder architecture, and innovatively introduces the window prompt and text prompt mechanism to construct a recognition module based on temporal features. The experimental results show that the proposed model achieves a detection F1 score of 83.66% on the NSTD dataset, and the character recognition correctness is 81.92%. In addition, the effectiveness of the cueing method is verified by ablation experiments: compared with the baseline model without cueing method, the precision rate, recall rate, F1 value, and average correct recognition rate based on the lexicon of the model are improved by 1.25%, 1.98%, 1.67%, and 2.57% respectively after the use of cueing method, which fully proves the effectiveness of the proposed method in the task of end-to-end Tibetan text detection and recognition.