Dynamic selection of ensemble technique for question answering systems: For Indian government dataset
摘要
This study introduces a novel strategy for improving a question-answering system by dynamically selecting the best ensemble technique among RoBERTa, BERT, ALBERT, DistilBERT, and ELECTRA language models. A unique metric based on the Average Confidence Score of these models is used to implement dynamic ensemble strategy selection. We test the proposed technique on JUDVLP-QADB.v1, a freshly created question-answering dataset from public Indian government domains. On the JUDVLP-QADB.v1 dataset, the technique improved Exact Match and F1 score significantly. Our ensemble technique selection experiments are two-step. First, we run the same group ensembles with varied language model settings. Second, we combine the best-performing ensembles from different groups to generate cross-group ensembles, and finally we present a novel system for dynamically selecting the ideal ensemble technique. The suggested technique chose the RoBERTa ensemble for the same group and the RoBERTa and DistilBERT ensemble for the cross group during the assessment. After dynamically picking the right ensemble model, we received a maximum Exact Match score of 69.01 and an F1-Score of 83.69. These values show a 15% and 10.31% increase above baseline models. Our study found that dynamically selecting ensembles based on a parameter improves question-answering system performance on the JUDVLP-QADB.v1 test dataset. Our novel ensemble selection method simplifies the procedure by lowering the difficulty of picking from