Cognitive-Aware Multi-modal Human-Swarm Interface for Optimal Collaboration
摘要
This paper presents the development of an interface that enables humans to collaborate with robot swarms (RSs) to effectively accomplish missions in highly uncertain environments. Autonomous RS technology has advanced to the point where robots can perform various tasks without human intervention. However, there are still two major uncertainties in mission execution. The first uncertainty arises from the limitations of artificial intelligence (AI), such as the inaccuracy of computer vision systems. The second uncertainty comes from the decision-making aspect, where the mission requirements may not be accurately communicated to the RS. To overcome these uncertainties, it is crucial to design a system that allows humans to monitor the RS and actively participate in critical decision-making processes during missions. We design an interface to: 1) estimate human cognitive states to monitor the human partner, 2) determine the best communication methods to calibrate cognitive states, and 3) allocate functions among humans and robots to optimize team performance. We demonstrate this approach through drone search and rescue (SAR) missions.