Recent advancements in the development of large language models (LLMs) have highlighted their remarkable capabilities in a range of reasoning and decision-related challenges. Nevertheless, the clarity and logical flow of their reasoning can still be enhanced through improved self-evaluation and reflective analysis. In this work, we propose Self-Assessing Chain-of-Draft (SACoD), an approach that allows LLMs to emulate a form of self-assessment during the reasoning process by employing dual Chain-of-Draft CoD thinking. This technique draws inspiration from human cognitive mechanisms, where the model produces concise yet meaningful intermediate outputs while addressing tasks. SACoD harnesses the potential of iterative thinking, wherein the model first generates an initial sequence of thoughts and then critically evaluates and distills these thoughts through a subsequent round of reasoning. This recursive strategy allows for more consistent, rational, and reliable responses, thereby enhancing the overall quality of decision-making at a significantly lower cost than the traditional Chain-of-Thought (CoT) thinking. We also demonstrate an effective integration of this methodology into existing LLM frameworks using simple prompt engineering. In this process, we achieved outcomes akin to that of the Learning-Refinement Model (LRM) without any extra training.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Revise to Precise: Self-assessing Chain-of-Draft for Robust Decision-Making in LLMs

  • Pratyay Banerjee,
  • Panthadeep Bhattacharjee,
  • Angshuman Jana

摘要

Recent advancements in the development of large language models (LLMs) have highlighted their remarkable capabilities in a range of reasoning and decision-related challenges. Nevertheless, the clarity and logical flow of their reasoning can still be enhanced through improved self-evaluation and reflective analysis. In this work, we propose Self-Assessing Chain-of-Draft (SACoD), an approach that allows LLMs to emulate a form of self-assessment during the reasoning process by employing dual Chain-of-Draft CoD thinking. This technique draws inspiration from human cognitive mechanisms, where the model produces concise yet meaningful intermediate outputs while addressing tasks. SACoD harnesses the potential of iterative thinking, wherein the model first generates an initial sequence of thoughts and then critically evaluates and distills these thoughts through a subsequent round of reasoning. This recursive strategy allows for more consistent, rational, and reliable responses, thereby enhancing the overall quality of decision-making at a significantly lower cost than the traditional Chain-of-Thought (CoT) thinking. We also demonstrate an effective integration of this methodology into existing LLM frameworks using simple prompt engineering. In this process, we achieved outcomes akin to that of the Learning-Refinement Model (LRM) without any extra training.