Boosting Q&A Generation for Military Equipment via Example Selection and Automated Prompt Engineering
摘要
Question and Answer Generation (QAG) is a text generation task that aims to generate contextually relevant questions and answers based on a given text. Due to the closed nature and difficulty in obtaining data in the military equipment field, relying entirely on domain experts to manually build datasets is too costly and difficult to implement effectively. In response, this paper proposes a QAG method based on example selection and automatic prompting engineering. This method utilizes the generative capabilities of large language models (LLMs), enhanced by high-quality examples and optimized prompts to improve reasoning and adaptability to problems. Initially, high-quality examples are selected from the training set documents based on a question-answer pair scoring strategy, ensuring coverage of different question types and complexities. Furthermore, the highest-scoring examples from each category are selected as prompt examples to further optimize the accuracy and relevance of question-answer generation. Additionally, by testing various LLMs, the most suitable model is chosen for task execution. Finally, we employ automatic prompting engineering to generate prompts, using the reasoning ability of LLMs to create an initial set of prompts and iteratively optimize to select the best prompts. Our method ranked second in the CCKS 2024 evaluation task 10, achieving a score of 91.5 \(\%\) .