Adapting Vision-Language Models for Hindi OCR
摘要
Optical Character Recognition (OCR) for Indian languages is challenging due to diverse scripts, complex characters, and varied writing styles. This work presents HindiOCR-VLM, a Vision-Language Model adapted for Hindi OCR using Low-Rank Adaptation (LoRA) on a model pre-trained for Chinese and English. We explore two adaptation strategies: (i) a single-stage model that directly predicts text at the page level—bypassing intermediate word or line detection—focused on printed documents, and (ii) a two-stage model for multi-domain scenarios (printed, handwritten, and scene text), which first detects words and then recognizes them individually. Inspired by human learning processes, we propose a progressive learning approach—a training strategy to the single-stage model to enhance language acquisition and accelerate convergence. Leveraging the vision encoder’s rich representations, our method enables effective multi-domain training. Experimental results and ablation studies show that HindiOCR-VLM handles the complexities of the Devanagari script well and outperforms domain-specific models, offering a unified and robust solution for Hindi OCR. Code and resources are available at: https://hindiocr-vlm.github.io/ .