<p>This study consists of qualitative empirical research conducted through exploratory tests with two large language models (LLMs): ChatGPT and Gemini. The methodology involved exploratory tests based on prompts designed with a probability question, using the well-documented “Linda Problem,” from cognitive psychology, and a newly developed “Mary Problem.” The analysis focuses on the outputs generated by each chatbot to assess whether their reasoning aligns with probability theory or is influenced by stereotypical descriptions in the prompts. The findings provide insights into how each chatbot handles logic and textual constructions, suggesting that, while both performed adequately on a well-known probabilistic problem, they exhibit significantly lower performance on novel tests requiring direct application of logic involving probabilities.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Exploratory analysis of logical and intuitive reasoning in large language models

  • Melise Peruchini,
  • Júlio Monteiro Teixeira

摘要

This study consists of qualitative empirical research conducted through exploratory tests with two large language models (LLMs): ChatGPT and Gemini. The methodology involved exploratory tests based on prompts designed with a probability question, using the well-documented “Linda Problem,” from cognitive psychology, and a newly developed “Mary Problem.” The analysis focuses on the outputs generated by each chatbot to assess whether their reasoning aligns with probability theory or is influenced by stereotypical descriptions in the prompts. The findings provide insights into how each chatbot handles logic and textual constructions, suggesting that, while both performed adequately on a well-known probabilistic problem, they exhibit significantly lower performance on novel tests requiring direct application of logic involving probabilities.