Large Language Models (LLMs) excel in open-domain tasks but struggle with knowledge hallucination and inefficient utilization in precision-critical scenarios. Existing approaches like Retrieval-Augmented Generation (RAG) rely on external knowledge, while static prompting lacks adaptability. We propose Self-Consistent Knowledge Generation (SCKG), a three-stage framework that combines meta-cognitive prompting and utility-driven optimization to activate LLMs’ intrinsic knowledge. SCKG first generates confidence-annotated knowledge candidates through introspective self-assessment, then dynamically selects optimal knowledge via a utility function balancing semantic relevance, contextual novelty, and self-evaluated confidence, and finally refines responses through iterative self-verification. Experiments on four datasets including Wizard-of-Wikipedia and MedChatZH demonstrate superior performance over vanilla baselines, achieving an improvement of 20.8% in BLEU and +28.5% in FActScore. This framework enhances LLMs’ reliability in knowledge-intensive generation tasks and offers a lightweight, self-supervised solution for optimizing intrinsic knowledge without external dependencies.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Self-consistent Knowledge Generation in Large Language Models: A Unified Framework of Meta-cognitive Prompting and Knowledge Utility Optimization

  • Sisi Peng,
  • Wenlin Zhang,
  • Shunhang Li,
  • Dan Qu

摘要

Large Language Models (LLMs) excel in open-domain tasks but struggle with knowledge hallucination and inefficient utilization in precision-critical scenarios. Existing approaches like Retrieval-Augmented Generation (RAG) rely on external knowledge, while static prompting lacks adaptability. We propose Self-Consistent Knowledge Generation (SCKG), a three-stage framework that combines meta-cognitive prompting and utility-driven optimization to activate LLMs’ intrinsic knowledge. SCKG first generates confidence-annotated knowledge candidates through introspective self-assessment, then dynamically selects optimal knowledge via a utility function balancing semantic relevance, contextual novelty, and self-evaluated confidence, and finally refines responses through iterative self-verification. Experiments on four datasets including Wizard-of-Wikipedia and MedChatZH demonstrate superior performance over vanilla baselines, achieving an improvement of 20.8% in BLEU and +28.5% in FActScore. This framework enhances LLMs’ reliability in knowledge-intensive generation tasks and offers a lightweight, self-supervised solution for optimizing intrinsic knowledge without external dependencies.