Despite recent deep neural network superhuman performance in many strategic board games, such as Chess and Go, there does not yet exist an algorithm that beats “Settlers of Catan” expert human players. Towards this direction, we present a combination of modern machine learning with a traditional tree-based adversarial search algorithm for initial settlement placement, and achieve performance that essentially matches the state-of-the-art. In particular, we use the n-player generalization of the classic Minimax search algorithm, known as Max \(^n\) , with the novelty that the evaluation function at the leaf nodes is the result of a forward pass in a trained convolutional neural network. Our work consists of two distinct parts that can work independently. The first is the use of the simple Max \(^n\) algorithm for the first time in this game setting. The second is the use of the neural network as an evaluation function for evaluating the initial settlement placement, and which could potentially be plugged into any adversarial search algorithm. After 10000 simulated games, which is a sufficient number to draw conclusions in this demanding strategic board game, we achieve performance close to the state-of-the-art; with the advantages that (a) our approach does not make use of any human-generated data corpus; and (b) that our approach’s runtime is acceptable by human players, and much lower than the state-of-the-art’s for initial placement in this domain.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Adversarial Search and Deep Learning for Strategic Settlement Placement in the “Settlers of Catan”

  • Diamantis Rafail Papadam,
  • Georgios Chalkiadakis

摘要

Despite recent deep neural network superhuman performance in many strategic board games, such as Chess and Go, there does not yet exist an algorithm that beats “Settlers of Catan” expert human players. Towards this direction, we present a combination of modern machine learning with a traditional tree-based adversarial search algorithm for initial settlement placement, and achieve performance that essentially matches the state-of-the-art. In particular, we use the n-player generalization of the classic Minimax search algorithm, known as Max \(^n\) , with the novelty that the evaluation function at the leaf nodes is the result of a forward pass in a trained convolutional neural network. Our work consists of two distinct parts that can work independently. The first is the use of the simple Max \(^n\) algorithm for the first time in this game setting. The second is the use of the neural network as an evaluation function for evaluating the initial settlement placement, and which could potentially be plugged into any adversarial search algorithm. After 10000 simulated games, which is a sufficient number to draw conclusions in this demanding strategic board game, we achieve performance close to the state-of-the-art; with the advantages that (a) our approach does not make use of any human-generated data corpus; and (b) that our approach’s runtime is acceptable by human players, and much lower than the state-of-the-art’s for initial placement in this domain.