Advancing Reasoning in Large Language Models: Promising Methods and Approaches
摘要
Large Language Models (LLMs) have demonstrated effectiveness in numerous natural language processing (NLP) tasks, yet their reasoning capabilities continue to present a substantial hurdle. Although LLMs exhibit remarkable fluency and factual recall, they frequently encounter difficulties with intricate reasoning, encompassing logical deduction, mathematical problem-solving, commonsense inference, and multistep reasoning. This survey provides a broad overview of recent techniques aimed at enhancing reasoning in LLMs. We classify these methods into principal categories, including prompting techniques (such as Chain-of-Thought reasoning, Self-Consistency, and Tree-of-Thought reasoning), modifications in architecture (like retrieval-augmented models, modular reasoning networks, and neuro-symbolic integration), and learning approaches (such as fine-tuning with datasets focused on reasoning, reinforcement learning, and self-supervised reasoning objectives). Additionally, we examine evaluation frameworks designed to measure reasoning in LLMs and highlight persistent issues, including hallucinations, robustness, and the generalization of reasoning across various tasks. By condensing recent advancements, this survey seeks to offer insights into promising avenues for future research and how reasoning can be effectively implemented within LLMs.