Exploratory analysis of logical and intuitive reasoning in large language models
摘要
This study consists of qualitative empirical research conducted through exploratory tests with two large language models (LLMs): ChatGPT and Gemini. The methodology involved exploratory tests based on prompts designed with a probability question, using the well-documented “Linda Problem,” from cognitive psychology, and a newly developed “Mary Problem.” The analysis focuses on the outputs generated by each chatbot to assess whether their reasoning aligns with probability theory or is influenced by stereotypical descriptions in the prompts. The findings provide insights into how each chatbot handles logic and textual constructions, suggesting that, while both performed adequately on a well-known probabilistic problem, they exhibit significantly lower performance on novel tests requiring direct application of logic involving probabilities.