A high precision and speed question answering system about the post-COVID-19
摘要
Coping with the mutation of COVID-19 and potential future pandemic pathogens is crucial in the post-COVID-19 era. The internet’s redundant data poses challenges for accurate and timely information retrieval, with current information retrieval systems often neglecting timeliness and speed. This paper proposes a Reader-Retriever architecture to design a question answering system with high accuracy and speed. We employ a distillation and fine-tuning process on a pre-trained model for the Reader pipeline, accomplished via joint training. Additionally, we propose a high-speed search algorithm IVF-PSQ within the Retriever pipeline. The system achieves state-of-the-art results with a Precision of 84.91%, an Exact Match (EM) score of 71.70%, a F1 score of 63.75%, a Recall of 51.04% and an average search time of 0.407 s.