Automated Processing of Historical Documents Using Named Entity Recognition and Face Extraction
摘要
This paper presents advancements in an automated document digitization pipeline, particularly tailored for historical archives from the Czech Institute for Study of Totalitarian Regimes. Key contributions include the enhancement of Named Entity Recognition (NER) and the introduction of a Face Extraction block. By leveraging modern large language models (LLMs), such as GPT-4o and its fine-tuned variants, the improved NER achieves promising accuracy in identifying entities such as persons, locations, and organizations. The Face Extraction block, based on two state-of-the-art models, RetinaFace for detection and ArcFace for recognition, enables the clustering and retrieval of documents featuring the same individuals. These innovations significantly streamline historians’ work by improving searchability and document organization. Experimental evaluations using the challenging NKVD dataset demonstrate the efficacy of the proposed pipeline.