Future autonomous aircraft need strong environment perception capabilities to perform challenging close-formation maneuvers like aerial refueling. RGB cameras combined with Machine Learning (ML)-based computer vision techniques offer a natural approach to achieve this. However, obtaining the required diverse training data is challenging in close-formation flight. One possible solution is to use synthetic images extracted from modern game-engines. Despite concerns about training ML models solely on synthetic images raised in other domains, this paper demonstrates that for the visually less complex aerial task of drogue detection during air-to-air refueling, synthetic images can suffice. After presenting a toolchain to generate synthetic images for the considered use-case, a state-of-the-art object detector is trained only on synthetic images. It achieves over 90% Average Precision on real-world evaluation images, showcasing synthetic images as a valid alternative for training. The results can help to advance ML-based environment perception in aviation and prompt new research on the sim-to-real domain gap.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Using Only Synthetic Images to Train a Drogue Detector for Aerial Refueling

  • Joachim Rüter,
  • Rebecca Schmidt

摘要

Future autonomous aircraft need strong environment perception capabilities to perform challenging close-formation maneuvers like aerial refueling. RGB cameras combined with Machine Learning (ML)-based computer vision techniques offer a natural approach to achieve this. However, obtaining the required diverse training data is challenging in close-formation flight. One possible solution is to use synthetic images extracted from modern game-engines. Despite concerns about training ML models solely on synthetic images raised in other domains, this paper demonstrates that for the visually less complex aerial task of drogue detection during air-to-air refueling, synthetic images can suffice. After presenting a toolchain to generate synthetic images for the considered use-case, a state-of-the-art object detector is trained only on synthetic images. It achieves over 90% Average Precision on real-world evaluation images, showcasing synthetic images as a valid alternative for training. The results can help to advance ML-based environment perception in aviation and prompt new research on the sim-to-real domain gap.