Question and Answer Generation (QAG) is a text generation task that aims to generate contextually relevant questions and answers based on a given text. Due to the closed nature and difficulty in obtaining data in the military equipment field, relying entirely on domain experts to manually build datasets is too costly and difficult to implement effectively. In response, this paper proposes a QAG method based on example selection and automatic prompting engineering. This method utilizes the generative capabilities of large language models (LLMs), enhanced by high-quality examples and optimized prompts to improve reasoning and adaptability to problems. Initially, high-quality examples are selected from the training set documents based on a question-answer pair scoring strategy, ensuring coverage of different question types and complexities. Furthermore, the highest-scoring examples from each category are selected as prompt examples to further optimize the accuracy and relevance of question-answer generation. Additionally, by testing various LLMs, the most suitable model is chosen for task execution. Finally, we employ automatic prompting engineering to generate prompts, using the reasoning ability of LLMs to create an initial set of prompts and iteratively optimize to select the best prompts. Our method ranked second in the CCKS 2024 evaluation task 10, achieving a score of 91.5 \(\%\) .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Boosting Q&A Generation for Military Equipment via Example Selection and Automated Prompt Engineering

  • Kunli Zhang,
  • Yongqi Zhu,
  • Yu Song,
  • Bohan Yu,
  • Guangyu Zhou,
  • Chenkang Zhu,
  • Pengcheng Wu

摘要

Question and Answer Generation (QAG) is a text generation task that aims to generate contextually relevant questions and answers based on a given text. Due to the closed nature and difficulty in obtaining data in the military equipment field, relying entirely on domain experts to manually build datasets is too costly and difficult to implement effectively. In response, this paper proposes a QAG method based on example selection and automatic prompting engineering. This method utilizes the generative capabilities of large language models (LLMs), enhanced by high-quality examples and optimized prompts to improve reasoning and adaptability to problems. Initially, high-quality examples are selected from the training set documents based on a question-answer pair scoring strategy, ensuring coverage of different question types and complexities. Furthermore, the highest-scoring examples from each category are selected as prompt examples to further optimize the accuracy and relevance of question-answer generation. Additionally, by testing various LLMs, the most suitable model is chosen for task execution. Finally, we employ automatic prompting engineering to generate prompts, using the reasoning ability of LLMs to create an initial set of prompts and iteratively optimize to select the best prompts. Our method ranked second in the CCKS 2024 evaluation task 10, achieving a score of 91.5 \(\%\) .