Evolving MCTS Macro-actions in Real-Time Domains
摘要
Forward Search techniques such as Monte Carlo Tree Search (MCTS) take advantage of the ability to query a simulator and see the results of taking certain actions. While they are very effective in many board games they remain relatively ineffective in real-time environments. This is often due to the prohibitive expense of building a sufficiently deep search tree for effective play in these environments. We address this by introducing a new Evolutionary Algorithm which evolves macro-actions for a particular domain. A macro-action is a sequence of actions treated as a single action by MCTS; domain appropriate macro-actions increase search tree depth and avoid unproductive search. Our technique evolves effective macro-actions that significantly improve MCTS performance in multiple real-time domains.