<p>Contemporary machine translation (MT) systems have achieved high semantic accuracy but systematically neglect emotional fidelity, a dimension critical for cross-cultural literary communication. From a cognitive computation standpoint, this neglect is not merely a stylistic deficiency: affective signals modulate memory consolidation, attention allocation, and communicative intent, rendering emotional fidelity a core rather than peripheral dimension of translation quality. Existing evaluation metrics—BLEU, METEOR, BERTScore, and COMET—lack explicit modelling of affective content preservation and are fundamentally reference-dependent, requiring high-quality human translations that are costly and often unavailable. To address these limitations, this study introduces the <b>Emotion Preservation Score (EMOS)</b>, a theoretically grounded, reference-free, multidimensional evaluation framework rooted in cognitive computation principles, which quantifies emotional fidelity in MT by directly comparing source and target emotion vectors without recourse to reference translations. Grounded in Ekman’s taxonomy of six basic emotions augmented with a neutrality dimension, EMOS integrates three complementary metrics derived from cognitive theory: Vector Similarity Score (VSS), which quantifies distributional similarity between seven-dimensional emotion vectors; Label Match Rate (LMR), which evaluates dominant emotional category preservation consistent with the primacy effect in emotional memory; and Emotional Diversity Ratio (EDR), which assesses emotional complexity retention via Shannon entropy, capturing the higher-order aesthetic processing most susceptible to cross-cultural degradation. Empirical weights (<InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\alpha = 0.50\)</EquationSource> </InlineEquation>, <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(\beta = 0.35\)</EquationSource> </InlineEquation>, <InlineEquation ID="IEq3"> <EquationSource Format="TEX">\(\gamma = 0.15\)</EquationSource> </InlineEquation>) were optimised through correlation analysis against human quality judgements on 500 parallel segments. The framework was validated on the CCL-SEL corpus—comprising 19,999 classical Chinese–English sentence pairs from nine canonical works—translated by DeepL, Google Translate, and GPT-4o. All three systems achieved good emotional fidelity (EMOS <InlineEquation ID="IEq4"> <EquationSource Format="TEX">\(> 0.75\)</EquationSource> </InlineEquation>), with GPT-4o demonstrating superior performance (EMOS <InlineEquation ID="IEq5"> <EquationSource Format="TEX">\(= 0.780\)</EquationSource> </InlineEquation>). EMOS exhibited stronger alignment with human judgements (VSS: <InlineEquation ID="IEq6"> <EquationSource Format="TEX">\(r = 0.79\)</EquationSource> </InlineEquation>; LMR: <InlineEquation ID="IEq7"> <EquationSource Format="TEX">\(r = 0.76\)</EquationSource> </InlineEquation>) than traditional metrics and successfully detected critical emotional distortions invisible to standard evaluation approaches, thereby establishing emotional fidelity as a distinct, quantifiable, and reference-free dimension of MT quality assessment with direct implications for cognitively plausible translation systems.</p>

EMOS: A Multidimensional Emotional Fidelity Evaluation Framework for Machine Translation Quality Assessment

期刊论文
出版模式：

开放获取
发表日期：

2026年6月1日

查看全文

Jingshi Zhou,
Fei Cheng,
Xiaojun Zhang

摘要

Contemporary machine translation (MT) systems have achieved high semantic accuracy but systematically neglect emotional fidelity, a dimension critical for cross-cultural literary communication. From a cognitive computation standpoint, this neglect is not merely a stylistic deficiency: affective signals modulate memory consolidation, attention allocation, and communicative intent, rendering emotional fidelity a core rather than peripheral dimension of translation quality. Existing evaluation metrics—BLEU, METEOR, BERTScore, and COMET—lack explicit modelling of affective content preservation and are fundamentally reference-dependent, requiring high-quality human translations that are costly and often unavailable. To address these limitations, this study introduces the Emotion Preservation Score (EMOS), a theoretically grounded, reference-free, multidimensional evaluation framework rooted in cognitive computation principles, which quantifies emotional fidelity in MT by directly comparing source and target emotion vectors without recourse to reference translations. Grounded in Ekman’s taxonomy of six basic emotions augmented with a neutrality dimension, EMOS integrates three complementary metrics derived from cognitive theory: Vector Similarity Score (VSS), which quantifies distributional similarity between seven-dimensional emotion vectors; Label Match Rate (LMR), which evaluates dominant emotional category preservation consistent with the primacy effect in emotional memory; and Emotional Diversity Ratio (EDR), which assesses emotional complexity retention via Shannon entropy, capturing the higher-order aesthetic processing most susceptible to cross-cultural degradation. Empirical weights ( \(\alpha = 0.50\) , \(\beta = 0.35\) , \(\gamma = 0.15\) ) were optimised through correlation analysis against human quality judgements on 500 parallel segments. The framework was validated on the CCL-SEL corpus—comprising 19,999 classical Chinese–English sentence pairs from nine canonical works—translated by DeepL, Google Translate, and GPT-4o. All three systems achieved good emotional fidelity (EMOS \(> 0.75\) ), with GPT-4o demonstrating superior performance (EMOS \(= 0.780\) ). EMOS exhibited stronger alignment with human judgements (VSS: \(r = 0.79\) ; LMR: \(r = 0.76\) ) than traditional metrics and successfully detected critical emotional distortions invisible to standard evaluation approaches, thereby establishing emotional fidelity as a distinct, quantifiable, and reference-free dimension of MT quality assessment with direct implications for cognitively plausible translation systems.