HyperPUT: Generating Synthetic Faulty Programs to Challenge Bug-Finding Tools

As research in automatically detecting bugs grows and produces new techniques, having suitable collections of programs with known bugs becomes crucial to reliably and meaningfully compare the effectiveness of these techniques. Most of the existing approaches rely on benchmarks collecting manually curated real-world bugs, or synthetic bugs seeded into real-world programs. Using real-world programs entails that extending the existing benchmarks or creating new ones remains a complex time-consuming task.
In this paper, we propose a complementary approach that automatically generates …

benchmarks bug bugs challenge collecting collections cs.cr cs.se real research suitable synthetic techniques tools world