A Southeast Asian Language OCR Dataset and Evaluation for Large Multimodal Models
摘要
This paper presents a comprehensive evaluation of the Optical Character Recognition (OCR) capabilities of state-of-the-art large multimodal models (LMMs) and traditional OCR models for Southeast Asian languages. Confronted by the critical lack of publicly available scene-text benchmarks, we construct a novel dataset comprising two distinct categories: real-world scene text images and single-line text images across four Southeast Asian languages—Vietnamese, Burmese, Thai, and Khmer, each meticulously annotated with precise text labels. Using this dataset, we assessed model performance with both image-captioning prompts and multilingual prompting strategies. The results revealed that overly detailed image descriptions negatively impact recognition accuracy, while using Southeast Asian language prompts leads to degraded model comprehension. This study provides valuable insights for enhancing the application of LMMs in Southeast Asian language scenarios. The dataset will be released later.