Multi-lingual Document Extraction and Validation Using Optical Character Recognition and Generative AI: Specific Focus on Arabic and English Texts
摘要
This paper investigates the challenges and solutions for extracting text from multi-lingual documents focusing on English and Arabic. The paper explores the limitations of traditional Optical Character Recognition (OCR) systems in handling multi-lingual documents and proposes a solution that combines OCR with Generative AI to enhance accuracy and context-awareness. It also explores challenges of handling bidirectional scripts, syntactic and semantic variances and proposes steps that leverage Large Language Model (LLM) and document layout understanding frameworks. The solution discovered Pytesseract and DocLing OCR tools and the Watsonx.ai LLM combination to solve the challenges posed in Arabic OCR systems due to the complexity of the Arabic script. An insurance case study is used to prove the solution and the model performance in terms of accuracy in extracting the text in English and Arabic. The solution uses pipeline architecture that integrates various stages like image preprocessing, language detection, OCR text extraction and Generative AI for post-processing, translation, content summarization and data structuring. Overall, the solution demonstrates significant advancements in multi-lingual document extraction in terms of accuracy, high effort savings, contextual understanding and adaptability.