<p>Metaphor is a pervasive rhetorical resource and a core cognitive mechanism for cross-domain conceptual mapping. We define metaphorical aptness as the extent to which a metaphorical expression captures the core attributes of the target concept, modeled as a continuous measure of how well source properties map onto the target in context. Although large language models (LLMs) have recently achieved strong results on metaphor detection, it remains unclear whether these gains reflect surface cue learning or human-like semantic integration. We propose an aptness-based continuum evaluation framework that extends binary identification into a finer-grained test of cognitive alignment. Experiments on the VU Amsterdam Metaphor Corpus (VUA), the largest English metaphor dataset, across representative LLMs (e.g., GPT-4, LLaMA-3) reveal a consistent gradient: performance is substantially higher on high-aptness metaphors than on low-aptness ones, suggesting models are more reliable when the mapping preserves central target properties but struggle when those attributes are weakly captured. We further replace binary labels with aptness annotations as supervision and show that a baseline trained under this paradigm improves metaphor detection over prior state-of-the-art systems. Overall, aptness provides an informative alignment signal and a principled basis for metaphor comprehension evaluation grounded in how metaphors convey core target attributes.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Rethinking Metaphor Evaluation: Aptness Judgments as a Cognitive Probe for Language Models

  • Cheng Yang,
  • Shiyi Lu,
  • Rui Sun,
  • Xingmao Zhang,
  • Qingbao Huang

摘要

Metaphor is a pervasive rhetorical resource and a core cognitive mechanism for cross-domain conceptual mapping. We define metaphorical aptness as the extent to which a metaphorical expression captures the core attributes of the target concept, modeled as a continuous measure of how well source properties map onto the target in context. Although large language models (LLMs) have recently achieved strong results on metaphor detection, it remains unclear whether these gains reflect surface cue learning or human-like semantic integration. We propose an aptness-based continuum evaluation framework that extends binary identification into a finer-grained test of cognitive alignment. Experiments on the VU Amsterdam Metaphor Corpus (VUA), the largest English metaphor dataset, across representative LLMs (e.g., GPT-4, LLaMA-3) reveal a consistent gradient: performance is substantially higher on high-aptness metaphors than on low-aptness ones, suggesting models are more reliable when the mapping preserves central target properties but struggle when those attributes are weakly captured. We further replace binary labels with aptness annotations as supervision and show that a baseline trained under this paradigm improves metaphor detection over prior state-of-the-art systems. Overall, aptness provides an informative alignment signal and a principled basis for metaphor comprehension evaluation grounded in how metaphors convey core target attributes.