In an era of pervasive misinformation, the help of artificial intelligence like Large Language Models (LLMs) in identifying and correcting falsehoods is critical. This study investigates the alignment between humans’ and GPT-4’s judgments on lying and falsity, with a focus on deceptive implicatures—statements that are literally true but imply something false. Using a cross-cultural human dataset (3660 participants across ten countries) as a benchmark, we evaluated GPT-4’s performance in six languages. Results indicate that while GPT-4’s judgments correlate significantly with human responses, notable discrepancies also exist. GPT-4 tends to attribute higher lie scores and lower falsity scores than humans and exhibits more severe moral judgments. These findings highlight the potential risks of deploying LLMs like GPT-4 for misinformation detection, suggesting possible over- or under classification (depending on the task). The study highlights the importance of cautious implementation and the need for further refinement to enhance human-LLM alignment in identifying deception.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

GPT-4’s Alignment with Human Lie and Falsity Attribution in Cases of Deceptive Implicatures

  • Nikolai Shurakov,
  • Moritz Kolbe,
  • Neele Engelmann,
  • Alex Wiegmann

摘要

In an era of pervasive misinformation, the help of artificial intelligence like Large Language Models (LLMs) in identifying and correcting falsehoods is critical. This study investigates the alignment between humans’ and GPT-4’s judgments on lying and falsity, with a focus on deceptive implicatures—statements that are literally true but imply something false. Using a cross-cultural human dataset (3660 participants across ten countries) as a benchmark, we evaluated GPT-4’s performance in six languages. Results indicate that while GPT-4’s judgments correlate significantly with human responses, notable discrepancies also exist. GPT-4 tends to attribute higher lie scores and lower falsity scores than humans and exhibits more severe moral judgments. These findings highlight the potential risks of deploying LLMs like GPT-4 for misinformation detection, suggesting possible over- or under classification (depending on the task). The study highlights the importance of cautious implementation and the need for further refinement to enhance human-LLM alignment in identifying deception.