This paper presents a comprehensive evaluation of the Optical Character Recognition (OCR) capabilities of state-of-the-art large multimodal models (LMMs) and traditional OCR models for Southeast Asian languages. Confronted by the critical lack of publicly available scene-text benchmarks, we construct a novel dataset comprising two distinct categories: real-world scene text images and single-line text images across four Southeast Asian languages—Vietnamese, Burmese, Thai, and Khmer, each meticulously annotated with precise text labels. Using this dataset, we assessed model performance with both image-captioning prompts and multilingual prompting strategies. The results revealed that overly detailed image descriptions negatively impact recognition accuracy, while using Southeast Asian language prompts leads to degraded model comprehension. This study provides valuable insights for enhancing the application of LMMs in Southeast Asian language scenarios. The dataset will be released later.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Southeast Asian Language OCR Dataset and Evaluation for Large Multimodal Models

  • Xu Yang,
  • Rui Chen,
  • Cunli Mao,
  • Ying Li,
  • Shengxiang Gao,
  • Zhengtao Yu

摘要

This paper presents a comprehensive evaluation of the Optical Character Recognition (OCR) capabilities of state-of-the-art large multimodal models (LMMs) and traditional OCR models for Southeast Asian languages. Confronted by the critical lack of publicly available scene-text benchmarks, we construct a novel dataset comprising two distinct categories: real-world scene text images and single-line text images across four Southeast Asian languages—Vietnamese, Burmese, Thai, and Khmer, each meticulously annotated with precise text labels. Using this dataset, we assessed model performance with both image-captioning prompts and multilingual prompting strategies. The results revealed that overly detailed image descriptions negatively impact recognition accuracy, while using Southeast Asian language prompts leads to degraded model comprehension. This study provides valuable insights for enhancing the application of LMMs in Southeast Asian language scenarios. The dataset will be released later.