With the widespread application of deep neural network (DNN) models, quality issues in deep learning (DL) systems have become increasingly prominent. One such issue is the insufficient test data, which fails to adequate test the decision-making behavior of DNN models. Existing research primarily focuses on fuzzing-based test data generation methods. However, these methods overlook the importance of seed queues and error-prone neurons, leading to a lack of diversity in the generated data and difficulty in revealing errors. Different from existing work, this paper proposes FTDG, a novel test data generation approach for DNN models. Firstly, FTDG designs the seed selection strategy based on model prediction uncertainty to create a seed list with diversity and high uncertainty. Secondly, we design the neuron selection strategy based on the relevance of model prediction errors to provide the necessary neurons for the mutation strategy. Finally, we iteratively mutate the seeds in the selected seed list until adversarial test data is generated. Experimental results show that FTDG outperforms the compared methods, with an average neuron coverage improvement of 5.1% to 94.1%, and an increase of at least 35.7% and 38.6% in the number and diversity of generated test data, respectively.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

FTDG: Fuzzing-Based Test Data Generation for Deep Neural Networks

  • Ziqi Chen,
  • Chuanqi Tao,
  • Tianzi Zang,
  • Hongjing Guo

摘要

With the widespread application of deep neural network (DNN) models, quality issues in deep learning (DL) systems have become increasingly prominent. One such issue is the insufficient test data, which fails to adequate test the decision-making behavior of DNN models. Existing research primarily focuses on fuzzing-based test data generation methods. However, these methods overlook the importance of seed queues and error-prone neurons, leading to a lack of diversity in the generated data and difficulty in revealing errors. Different from existing work, this paper proposes FTDG, a novel test data generation approach for DNN models. Firstly, FTDG designs the seed selection strategy based on model prediction uncertainty to create a seed list with diversity and high uncertainty. Secondly, we design the neuron selection strategy based on the relevance of model prediction errors to provide the necessary neurons for the mutation strategy. Finally, we iteratively mutate the seeds in the selected seed list until adversarial test data is generated. Experimental results show that FTDG outperforms the compared methods, with an average neuron coverage improvement of 5.1% to 94.1%, and an increase of at least 35.7% and 38.6% in the number and diversity of generated test data, respectively.