Real-Time Resume Screening Using Kafka Based Applicant Tracking System
摘要
Recruitment is a fundamental yet resource intensive process for organizations, especially when handling large volumes of applications. Manually screening resumes to identify qualified candidates is time consuming, error prone, and difficult to scale. Human judgment may overlook suitable candidates or inconsistently apply criteria, resulting in delayed hiring and reduced efficiency. Traditional systems also struggle to keep up with the dynamic demands of large scale recruitment workflows. This paper suggests a distributed, real time ATS for automating resume ingestion, parsing, and ranking. The system, built with Apache Kafka and Spark Streaming, eliminates rule based static filtering by taking advantage of RAKE, regular expressions, and TF-IDF scoring to extract and score structured resume data. The pipeline is automated from file ingestion to candidate ranking while offloading workloads onto multiple FastAPI based microservices. A load balancer based on hashing distributes resumes equally among processing servers. A resume is parsed to candidate details, and these are published to Kafka topics. Candidate resumes are scored against job descriptions using TF-IDF scoring. This allows unbiased, real-time candidate evaluation for multiple positions. Kafka achieves high availability with topic replication and offset management, and Spark Streaming achieves continuous processing and recovery through checkpointing. Structured outputs in JSON format enable downstream analytics and integration with external systems. Performance measurements show the system processes growing volumes of resumes with invariant latency and high performance. Fault tolerance controls also guarantee seamless operation in the event of server or network failure. Scalable performance is verified by complexity analysis for different job types and resume contents. With automated resume screening through distributed stream processing, this system eliminates most human effort, reduces hiring time, and offers a fault resilient, data driven alternative to conventional ATS implementations.