A variety of biases and heuristics shape human decision-making. When training artificial reasoning systems on corpora that include their use, the decisions made by these systems may then reflect these biases and heuristics. The work presented here explores the extent to which the phenomenon of probability matching is present in decisions made by GPT-4o mini. Results show no clear evidence of probability matching nor an optimal maximizing strategy. Instead, GPT-4o mini’s behavior is consistent with previous results and shows an inability to perceive and reason over the base frequencies accurately. Still, there appears to be a compounding effect of domain-related biases about probabilities and whether frequencies are presented as summarized counts or as individual event outcomes. This behavior is plausibly due to patterns in the training data in the former case and limitations of counting and reasoning via statistical association in the latter.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Exploring Cognitive Biases in LLM Predictions: Probability Matching in GPT-4o Mini

  • Erin Zaroukian

摘要

A variety of biases and heuristics shape human decision-making. When training artificial reasoning systems on corpora that include their use, the decisions made by these systems may then reflect these biases and heuristics. The work presented here explores the extent to which the phenomenon of probability matching is present in decisions made by GPT-4o mini. Results show no clear evidence of probability matching nor an optimal maximizing strategy. Instead, GPT-4o mini’s behavior is consistent with previous results and shows an inability to perceive and reason over the base frequencies accurately. Still, there appears to be a compounding effect of domain-related biases about probabilities and whether frequencies are presented as summarized counts or as individual event outcomes. This behavior is plausibly due to patterns in the training data in the former case and limitations of counting and reasoning via statistical association in the latter.