This chapter looks closer at the exploration-exploitation decision. Given its importance, most living organisms have developed mechanisms incentivizing exploration. One way of motivating exploration is through the introduction of higher-level rewards for exploration activities. The chapter refers to it as propensity for exploration (PEX). Curiosity and boredom, evident in humans as well as animals, are emotions that facilitate such rewards. Data scientists refer to such exogenous objects as hyper-parameters. While data scientists could change a hyper-parameter and retrain their algorithm, a person cannot alter their genetic predispositions and relive their life. As a result, humans have developed a more flexible mechanism for fine-tuning their PEX—the adoption of biased perceptions of reality. Building on recent research, the chapter introduces the ABC-model of exploration. It states that the likelihood for the exploration of an option increases with the quantity (PEX*β) * (B—C), where B is the expected benefit of exploration, C is the expected cost of exploration, PEX is one’s inherited PEX, and β is a belief bias parameter regarding the expected net benefits of exploration. When β > 1, people would explore more; and when β < 1, people would explore less. The chapter also argues that exploration tends to be under-incentivized genetically. As a result, most cultures tend to develop beliefs encouraging exploration. To be an effective correction mechanism, these beliefs need to be biased.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Born to Explore

  • Christo A. Pirinsky

摘要

This chapter looks closer at the exploration-exploitation decision. Given its importance, most living organisms have developed mechanisms incentivizing exploration. One way of motivating exploration is through the introduction of higher-level rewards for exploration activities. The chapter refers to it as propensity for exploration (PEX). Curiosity and boredom, evident in humans as well as animals, are emotions that facilitate such rewards. Data scientists refer to such exogenous objects as hyper-parameters. While data scientists could change a hyper-parameter and retrain their algorithm, a person cannot alter their genetic predispositions and relive their life. As a result, humans have developed a more flexible mechanism for fine-tuning their PEX—the adoption of biased perceptions of reality. Building on recent research, the chapter introduces the ABC-model of exploration. It states that the likelihood for the exploration of an option increases with the quantity (PEX*β) * (B—C), where B is the expected benefit of exploration, C is the expected cost of exploration, PEX is one’s inherited PEX, and β is a belief bias parameter regarding the expected net benefits of exploration. When β > 1, people would explore more; and when β < 1, people would explore less. The chapter also argues that exploration tends to be under-incentivized genetically. As a result, most cultures tend to develop beliefs encouraging exploration. To be an effective correction mechanism, these beliefs need to be biased.