Scholarlink

The development of the Advanced Air Mobility(AAM) solution has highlighted the necessity to design automatic recognition and tracking systems of non-collaborative intruders for collision avoidance. To robust our detection effort, we decided to fuse the data coming from a COTS fmcw-MIMO RADAR system with the image produced by an IR camera, creating a system that can be used during the day, night, and it is ready to be tested in a Degraded Visual Environment(DVE). Current study leverages advanced simulation methodologies and relies on Ansys AVxcelerate® allowing to replicate multiple sensor output in the modeled scenario and to take into account the multi-spectral nature of the physics involved by assigning thermal footprints, emissivity and material(s) properties for all entities involved in the simulation. The sensor output from simulation has been interfaced with a widely diffused robotic middleware (ROS2) allowing a rapid development of the perception pipelines detailed in this work and heading to a full SIL capability to test algorithms under one-to-one correspondence with real sensors output. Regarding LWIR thermal cameras both the environment and the entities represented in the simulation were created by applying appropriate thermal meshes and assigning the reflectivity and emissivity characteristics of the modeled materials in the simulation. This allowed for the most realistic possible reproduction of the sensor’s output in terms of thermal maps. Subsequently, an Automatic Gain Control (ACG) algorithm, developed by us and briefly summarized in this work, was applied to obtain a thermal image suitable for the development of perception algorithms. Similarly, conductivity, permeability, permittivity, and roughness, were assigned to each object in the scene to accurately simulate RADAR beams reflection. Advanced signal processing techniques were employed to analyze the raw RADAR output data and estimate both relative speed and a detailed 3D point cloud, allowing for the isolation of non-collaborative intruders from the environment. The proposed fusion technique employs the re-projection of the isolated RADAR tracks into the the camera plane to associate the semantic information applying, at latter time, a tracking algorithm for all detected entities.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A Simulation Study on Thermal Camera and FMCW RADAR Sensor Fusion for UAV Sense and Avoid in the Context of AAM

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Mattia Bertolo,
  • Riccardo Brivio,
  • Matteo Baiguera,
  • Salvatore Sandullo

摘要

The development of the Advanced Air Mobility(AAM) solution has highlighted the necessity to design automatic recognition and tracking systems of non-collaborative intruders for collision avoidance. To robust our detection effort, we decided to fuse the data coming from a COTS fmcw-MIMO RADAR system with the image produced by an IR camera, creating a system that can be used during the day, night, and it is ready to be tested in a Degraded Visual Environment(DVE). Current study leverages advanced simulation methodologies and relies on Ansys AVxcelerate® allowing to replicate multiple sensor output in the modeled scenario and to take into account the multi-spectral nature of the physics involved by assigning thermal footprints, emissivity and material(s) properties for all entities involved in the simulation. The sensor output from simulation has been interfaced with a widely diffused robotic middleware (ROS2) allowing a rapid development of the perception pipelines detailed in this work and heading to a full SIL capability to test algorithms under one-to-one correspondence with real sensors output. Regarding LWIR thermal cameras both the environment and the entities represented in the simulation were created by applying appropriate thermal meshes and assigning the reflectivity and emissivity characteristics of the modeled materials in the simulation. This allowed for the most realistic possible reproduction of the sensor’s output in terms of thermal maps. Subsequently, an Automatic Gain Control (ACG) algorithm, developed by us and briefly summarized in this work, was applied to obtain a thermal image suitable for the development of perception algorithms. Similarly, conductivity, permeability, permittivity, and roughness, were assigned to each object in the scene to accurately simulate RADAR beams reflection. Advanced signal processing techniques were employed to analyze the raw RADAR output data and estimate both relative speed and a detailed 3D point cloud, allowing for the isolation of non-collaborative intruders from the environment. The proposed fusion technique employs the re-projection of the isolated RADAR tracks into the the camera plane to associate the semantic information applying, at latter time, a tracking algorithm for all detected entities.