Shifting fuzzing left in software workflows
摘要
Fuzzing has proven to be an effective tool for finding bugs in software, with Google’s OSSFuzz alone being responsible for finding thousands of critical security vulnerabilities in open source projects. As software development practices evolve, there is a growing recognition of the need to integrate security testing earlier in the development process. Yet in our survey of software practitioners, only 20% used fuzzing as part of their development workflow. In this paper, we explore how fuzzing can fit into the software development life cycle. We do so with two empirical studies, including perspectives from over 40 industry practitioners. First, in a survey of software professionals, we identify several gaps between current state-of-the-art fuzzers’ capabilities and engineers’ expectations. In particular, we find that most developers are willing to use fuzzers, but prefer shorter, more frequent fuzzing runs as part of a continuous integration/continuous deployment (CI/CD) workflow. Next, based on results of this survey, we assess state-of-the-art fuzzers’ capabilities in the context of CI/CD and local development workflows. We observe that existing fuzzers can find up to 50% of bugs within 5 minutes of fuzzing, meeting developer expectations from our survey, but that further research is needed to uncover more difficult bugs within tolerable time limits. Additionally, we see that the initial corpus and time needed to build and analyze a project for a particular fuzzer both have a significant effect on fuzzer effectiveness in this context. We hope that our work will help the community to drive wider adoption of fuzzing in the software development lifecycle.