Choose Your Programming Copilot: A Comparison of the Program Synthesis Performance of GitHub Copilot and Genetic Programming

Created by W.Langdon from gp-bibliography.bib Revision:1.7630

@Misc{DBLP:journals/corr/abs-2111-07875,

• author = "Dominik Sobania and Martin Briesch and Franz Rothlauf",
• title = "Choose Your Programming Copilot: A Comparison of the Program Synthesis Performance of {GitHub} Copilot and Genetic Programming",
• howpublished = "arXiv",
• volume = "abs/2111.07875",
• year = "2021",
• month = "15 " # nov,
• keywords = "genetic algorithms, genetic programming, Program Synthesis, Genetic Programming, Large-Scale Language Models, Codex, GitHub Copilot, Software Engineering",
• URL = "https://arxiv.org/abs/2111.07875",
• eprinttype = "arXiv",
• eprint = "2111.07875",
• timestamp = "Tue, 16 Nov 2021 00:00:00 +0100",
• biburl = "https://dblp.org/rec/journals/corr/abs-2111-07875.bib",
• bibsource = "dblp computer science bibliography, https://dblp.org",
• size = "9 pages",
• abstract = "GitHub Copilot, an extension for the Visual Studio Code development environment powered by the large-scale language model Codex, makes automatic program synthesis available for software developers. This model has been extensively studied in the field of deep learning, however, a comparison to genetic programming, which is also known for its performance in automatic program synthesis, has not yet been carried out. In this paper, we evaluate GitHub Copilot on standard program synthesis benchmark problems and compare the achieved results with those from the genetic programming literature. In addition, we discuss the performance of both approaches. We find that the performance of the two approaches on the benchmark problems is quite similar, however, in comparison to GitHub Copilot, the program synthesis approaches based on genetic programming are not yet mature enough to support programmers in practical software development. Genetic programming usually needs a huge amount of expensive hand-labelled training cases and takes too much time to generate solutions. Furthermore, source code generated by genetic programming approaches is often bloated and difficult to understand. For future work on program synthesis with genetic programming, we suggest researchers to focus on improving the execution time, readability, and usability.",

}

Genetic Programming entries for Dominik Sobania Martin Briesch Franz Rothlauf

Citations