Automatic Program Repair Using Genetic Programming

Software quality is an urgent problem. There are so many bugs in industrial program source code that mature software projects are known to ship with both known and unknown bugs, and the number of outstanding defects typically exceeds the resources available to address them. This has become a pressing economic problem whose costs in the United States can be measured in the billions of dollars annually.

A dominant reason that software defects are so expensive is that fixing them remains a manual process. The process of identifying, triaging, reproducing, and localizing a particular bug, coupled with the task of understanding the underlying error, identifying a set of code changes that address it correctly, and then verifying those changes, costs both time and money, and the cost of repairing a defect can increase by orders of magnitude as development progresses. As a result, many defects, including critical security defects, remain unaddressed for long periods of time. Moreover, humans are error-prone, and many human fixes are imperfect, in that they are either incorrect or lead to crashes, hangs, corruption, or security problems. As a result, defect repair has become a major component of software maintenance, which in turn consumes up to 90% of the total lifecycle cost of a given piece of software.

Although considerable research attention has been paid to supporting various aspects of the manual debugging process, and also to preempting or dynamically addressing particular classes of vulnerabilities, such as buffer overruns, there exist virtually no previous automated solutions that address the synthesis of patches for general bugs as they are reported in real-world software.

The primary contribution of this dissertation is GenProg, one of the very first automatic solutions designed to help alleviate the manual bug repair burden by automatically and generically patching bugs in deployed and legacy software. GenProg uses a novel genetic programming algorithm, guided by test cases and domain-specific operators, to affect scalable, expressive, and high quality automated repair. We present experimental evidence to substantiate our claims that GenProg can repair multiple types of bugs in multiple types of programs, and that it can repair a large proportion of the bugs that human developers address in practice (that it is expressive); that it scales to real-world system sizes (that it is scalable); and that it produces repairs that are of sufficiently high quality. Over the course of this evaluation, we contribute new benchmark sets of real bugs in real open-source software and novel experimental frameworks for quantitatively evaluating an automated repair technique. We also contribute a novel characterization of the automated repair search space, and provide analysis both of that space and of the performance and scaling behavior of our technique.

General automated software repair was unheard of in 2009. In 2013, it has its own multi-paper sessions in top tier software engineering conferences. The research area shows no signs of slowing down. This dissertation’s description of GenProg provides a detailed report on the state of the art for early automated software repair efforts.