Scholarlink

Fuzz testing is a widely adopted testing methodology in software engineering that offers efficient means of testing software and identifying vulnerabilities. This paper presents a universal framework aimed at improving the efficiency of fuzz testing for Autonomous Systems (AS), particularly Unmanned Aerial Vehicle (UAV) autonomous systems. At its core is SaFliTe (Safe Flight Testing), a predictive component that evaluates whether a test case meets predefined safety criteria. By leveraging the large language model (LLM) with information about the test objective and the AS state, SaFliTe assesses the relevance of each test case. We evaluated SaFliTe by instantiating it with various LLMs, including GPT-3.5, Mistral-7B, and Llama2-7B, and integrating it into four fuzz testing tools: PGFuzz, DeepHyperion-UAV, CAMBA, and TUMB. These tools are designed specifically for testing autonomous drone control systems. The experimental results demonstrate that, compared to PGFuzz, SaFliTe increased the likelihood of selecting operations that triggered bug occurrences in each fuzzing iteration by an average of 93.1%. Additionally, after integrating SaFliTe, the ability of DeepHyperion-UAV, CAMBA, and TUMB to generate test cases that caused system safety violations increased by 234.5%, 33.3%, and 17.8%, respectively. The benchmark used in evaluation was from CPS-UAV Tool Competition 2024.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

SaFliTe: Fuzzing Autonomous Systems via Large Language Models

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Taohong Zhu,
  • Adrians Skapars,
  • Fardeen Mackenzie,
  • Declan Kehoe,
  • William Newton,
  • Suzanne Embury,
  • Youcheng Sun

摘要

Fuzz testing is a widely adopted testing methodology in software engineering that offers efficient means of testing software and identifying vulnerabilities. This paper presents a universal framework aimed at improving the efficiency of fuzz testing for Autonomous Systems (AS), particularly Unmanned Aerial Vehicle (UAV) autonomous systems. At its core is SaFliTe (Safe Flight Testing), a predictive component that evaluates whether a test case meets predefined safety criteria. By leveraging the large language model (LLM) with information about the test objective and the AS state, SaFliTe assesses the relevance of each test case. We evaluated SaFliTe by instantiating it with various LLMs, including GPT-3.5, Mistral-7B, and Llama2-7B, and integrating it into four fuzz testing tools: PGFuzz, DeepHyperion-UAV, CAMBA, and TUMB. These tools are designed specifically for testing autonomous drone control systems. The experimental results demonstrate that, compared to PGFuzz, SaFliTe increased the likelihood of selecting operations that triggered bug occurrences in each fuzzing iteration by an average of 93.1%. Additionally, after integrating SaFliTe, the ability of DeepHyperion-UAV, CAMBA, and TUMB to generate test cases that caused system safety violations increased by 234.5%, 33.3%, and 17.8%, respectively. The benchmark used in evaluation was from CPS-UAV Tool Competition 2024.