Creating Photorealistic Landscapes for Robot Path Planning with Iterative Prompting
摘要
This paper presents an approach to create photorealistic landscapes for simulating robots in unknown environments using latent diffusion and generative adversarial network (GAN)-enabled AI software. Such problems arise when robotic exploration planning is conducted before any detailed investigation of the terrain, either due to the urgency of the mission or to the impossibility of acquiring prior information. Examples include exploration of newly discovered cave corridors or planning of future lander missions to the moons in the outer Solar System, where the terrain topography remains unknown until the actual landing. Any prior knowledge of the physical conditions and feasible terrain features in the target environment is used to define prompts for image generation with online software such as Leonardo.ai Stable Diffusion. So generated images are evaluated using a supervised learning procedure, where assessment criteria values serve as labels to iteratively refine the prompts. After a few initial supervised steps, this procedure can transition to a semi-supervised mode. In this phase, image features are extracted using segmentation and other image processing techniques and assessed automatically according to a predefined scheme. The process stops when the generated landscape meets the desired criteria or after a given number of iterations. The resulting images are then used as backgrounds for simulations involving exploration site identification, robot path planning and navigation, as well as cooperation and coordination. As an example, we will demonstrate how landscapes generated with Stable Diffusion could be used as background in simulations of multi-robot exploration missions to Jupiter's and Saturn’s moons.