Choose Your Programming Copilot: A Comparison of the Program Synthesis Performance of GitHub Copilot and Genetic Programming
Created by W.Langdon from
gp-bibliography.bib Revision:1.8098
- @InProceedings{sobania:2022:GECCO,
-
author = "Dominik Sobania and Martin Briesch and
Franz Rothlauf",
-
title = "Choose Your Programming Copilot: A Comparison of the
Program Synthesis Performance of {GitHub} Copilot and
Genetic Programming",
-
booktitle = "Proceedings of the 2022 Genetic and Evolutionary
Computation Conference",
-
year = "2022",
-
editor = "Alma Rahat and Jonathan Fieldsend and
Markus Wagner and Sara Tari and Nelishia Pillay and Irene Moser and
Aldeida Aleti and Ales Zamuda and Ahmed Kheiri and
Erik Hemberg and Christopher Cleghorn and Chao-li Sun and
Georgios Yannakakis and Nicolas Bredeche and
Gabriela Ochoa and Bilel Derbel and Gisele L. Pappa and
Sebastian Risi and Laetitia Jourdan and
Hiroyuki Sato and Petr Posik and Ofer Shir and Renato Tinos and
John Woodward and Malcolm Heywood and Elizabeth Wanner and
Leonardo Trujillo and Domagoj Jakobovic and
Risto Miikkulainen and Bing Xue and Aneta Neumann and
Richard Allmendinger and Inmaculada Medina-Bulo and
Slim Bechikh and Andrew M. Sutton and
Pietro Simone Oliveto",
-
pages = "1019--1027",
-
address = "Boston, USA",
-
series = "GECCO '22",
-
month = "9-13 " # jul,
-
organisation = "SIGEVO",
-
publisher = "Association for Computing Machinery",
-
publisher_address = "New York, NY, USA",
-
keywords = "genetic algorithms, genetic programming, program
synthesis, large-scale language models, GitHub copilot,
software engineering, codex",
-
isbn13 = "978-1-4503-9237-2",
-
DOI = "doi:10.1145/3512290.3528700",
-
abstract = "GitHub Copilot, an extension for the Visual Studio
Code development environment powered by the large-scale
language model Codex, makes automatic program synthesis
available for software developers. This model has been
extensively studied in the field of deep learning,
however, a comparison to genetic programming, which is
also known for its performance in automatic program
synthesis, has not yet been carried out. In this paper,
we evaluate GitHub Copilot on standard program
synthesis benchmark problems and compare the achieved
results with those from the genetic programming
literature. In addition, we discuss the performance of
both approaches. We find that the performance of the
two approaches on the benchmark problems is quite
similar, however, in comparison to GitHub Copilot, the
program synthesis approaches based on genetic
programming are not yet mature enough to support
programmers in practical software development. Genetic
programming usually needs a huge amount of expensive
hand-labeled training cases and takes too much time to
generate solutions. Furthermore, source code generated
by genetic programming approaches is often bloated and
difficult to understand. For future work on program
synthesis with genetic programming, we suggest
researchers to focus on improving the execution time,
readability, and usability.",
-
notes = "GECCO-2022 A Recombination of the 31st International
Conference on Genetic Algorithms (ICGA) and the 27th
Annual Genetic Programming Conference (GP)",
- }
Genetic Programming entries for
Dominik Sobania
Martin Briesch
Franz Rothlauf
Citations