Cognitive Reasoning in Translation: Evaluating Chain-of-Thought, Explaining, Metacognition, and Critique in Humans and General-Purpose vs. Advanced-Reasoning Large Language Models
摘要
Cognitive reasoning is essential to translation, shaping meaning preservation, cultural adaptation, and authenticity. We compared four reasoning approaches—chain-of-thought, explaining, metacognition, and critique/refining—using two large language models (LLMs): a general-purpose model (GPT-4o) and an advanced reasoning model (GPT-o1). Across all approaches, we observed frequent unfaithful reasoning, as reasoning quality scores did not reliably align with translation quality. GPT-o1 produced better translations overall, likely due to its ability to perform multi-step reasoning internally without needing to verbalize it. In contrast, GPT-4o consistently generated higher-quality explicit reasoning, drawing on its broad world knowledge—except in the chain-of-thought condition, where GPT-o1’s internal reasoning aligned well with the prompt and yielded strong explicit reasoning outputs. However, GPT-4o’s superior reasoning quality did not translate into better translation outcomes due to persistent unfaithfulness. Human evaluators highlighted significant strengths and weaknesses across the reasoning approaches. Key advantages included human-like reasoning patterns, in-depth and relevant analyses, balanced multi-level and multi-aspect evaluations, and effective critiques and refinements. Conversely, limitations included inadequate high-level, top-down analysis typical of human reasoning, unrealistic assumptions regarding audience knowledge, omission of critical linguistic phenomena, rigid adherence to rules leading to unnatural translations, insufficient contextual support, disproportionate focus on complex terms at the expense of broader context, inadequate critical evaluations, and occasionally misguided critiques resulting in poorer translation outcomes. These insights underline the complexity of aligning cognitive reasoning quality with translation fidelity in LLM-driven translation tasks.