FTDG: Fuzzing-Based Test Data Generation for Deep Neural Networks
摘要
With the widespread application of deep neural network (DNN) models, quality issues in deep learning (DL) systems have become increasingly prominent. One such issue is the insufficient test data, which fails to adequate test the decision-making behavior of DNN models. Existing research primarily focuses on fuzzing-based test data generation methods. However, these methods overlook the importance of seed queues and error-prone neurons, leading to a lack of diversity in the generated data and difficulty in revealing errors. Different from existing work, this paper proposes FTDG, a novel test data generation approach for DNN models. Firstly, FTDG designs the seed selection strategy based on model prediction uncertainty to create a seed list with diversity and high uncertainty. Secondly, we design the neuron selection strategy based on the relevance of model prediction errors to provide the necessary neurons for the mutation strategy. Finally, we iteratively mutate the seeds in the selected seed list until adversarial test data is generated. Experimental results show that FTDG outperforms the compared methods, with an average neuron coverage improvement of 5.1% to 94.1%, and an increase of at least 35.7% and 38.6% in the number and diversity of generated test data, respectively.