In the development of autonomous systems (AS), perception components play a crucial role in the interpretation of environmental information. However, when these components misinterpret data, it can lead to serious or even fatal system-level failures. Unfortunately, current testing methods for perception components show significant limitations in generating test data that effectively translates into real-world performance and in capturing the diversity of rare but safety-critical long-tail scenarios. These limitations are particularly pronounced in LiDAR sensor-based perception systems, as such sensors provide detailed 3D scans of the world and operate reliably under various lighting conditions, making them indispensable components of modern AS. To overcome these challenges, we have developed an innovative testing tool, called LiDWeather, specifically for evaluating LiDAR-based perception systems. This tool makes use of existing real-world data and introduces carefully designed weather transformation operators to generate realistic and diverse test cases. In addition, we employ a fitness-guided algorithm to identify weather perturbations that significantly affect the performance of perception systems. To evaluate the effectiveness of this tool, we conduct a series of experiments using a 3D semantic segmentation model. The experimental results demonstrate that our tool efficiently generates test cases revealing potential errors in AS, providing a strong guarantee for the reliability and safety of AS.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Fitness-Guided Point Cloud Weather Synthesis for Testing Autonomous Systems

  • Jixiang Zhou,
  • Xiaoning Ren,
  • Chongyang Liu

摘要

In the development of autonomous systems (AS), perception components play a crucial role in the interpretation of environmental information. However, when these components misinterpret data, it can lead to serious or even fatal system-level failures. Unfortunately, current testing methods for perception components show significant limitations in generating test data that effectively translates into real-world performance and in capturing the diversity of rare but safety-critical long-tail scenarios. These limitations are particularly pronounced in LiDAR sensor-based perception systems, as such sensors provide detailed 3D scans of the world and operate reliably under various lighting conditions, making them indispensable components of modern AS. To overcome these challenges, we have developed an innovative testing tool, called LiDWeather, specifically for evaluating LiDAR-based perception systems. This tool makes use of existing real-world data and introduces carefully designed weather transformation operators to generate realistic and diverse test cases. In addition, we employ a fitness-guided algorithm to identify weather perturbations that significantly affect the performance of perception systems. To evaluate the effectiveness of this tool, we conduct a series of experiments using a 3D semantic segmentation model. The experimental results demonstrate that our tool efficiently generates test cases revealing potential errors in AS, providing a strong guarantee for the reliability and safety of AS.