- author = "Dominik Sobania and Franz Rothlauf",
- title = "A Generalizability Measure for Program Synthesis with Genetic Programming",
- booktitle = "Proceedings of the 2021 Genetic and Evolutionary Computation Conference",
- year = "2021",
- editor = "Francisco Chicano and Alberto Tonda and Krzysztof Krawiec and Marde Helbig and Christopher W. Cleghorn and Dennis G. Wilson and Georgios Yannakakis and Luis Paquete and Gabriela Ochoa and Jaume Bacardit and Christian Gagne and Sanaz Mostaghim and Laetitia Jourdan and Oliver Schuetze and Petr Posik and Carlos Segura and Renato Tinos and Carlos Cotta and Malcolm Heywood and Mengjie Zhang and Leonardo Trujillo and Risto Miikkulainen and Bing Xue and Aneta Neumann and Richard Allmendinger and Fuyuki Ishikawa and Inmaculada Medina-Bulo and Frank Neumann and Andrew M. Sutton",
- pages = "822--829",
- address = "internet",
- series = "GECCO '21",
- month = jul # " 10-14",
- organisation = "SIGEVO",
- publisher = "Association for Computing Machinery",
- publisher_address = "New York, NY, USA",
- note = "best paper",
- keywords = "genetic algorithms, genetic programming, SBSE, search-based software engineering, Program synthesis, Generalidation, Software engineering",
- isbn13 = "9781450383509",
-
DOI = "
doi:10.1145/3449639.3459305",
- size = "8 pages",
- abstract = "The generalisability of programs synthesized by genetic programming (GP) to unseen test cases is one of the main challenges of GP-based program synthesis. Recent work showed that increasing the amount of training data improves the generalizability of the programs synthesized by GP. However, generating training data is usually an expensive task as the output value for every training case must be calculated manually by the user. Therefore, this work suggests an approximation of the expected generalization ability of solution candidates found by GP. To obtain candidate solutions that all solve the training cases, but are structurally different, a GP run is not stopped after the first solution is found that solves all training instances but search continues for more generations. For all found candidate solutions (solving all training cases), we calculate the behavioural vector for a set of randomly generated additional inputs. The proportion of the number of different found candidate solutions generating the same behavioral vector with highest frequency compared to all other found candidate solutions with different behavior can serve as an approximation for the generalizability of the found solutions. We present experimental results for a number of standard program synthesis problems confirming the high prediction accuracy.",
-
notes = "Best paper awarded video
https://www.youtube.com/watch?v=jrT0sfq6WjM 56:42 --
57:14
University of Mainz, Germany.
GECCO-2021 A Recombination of the 30th International Conference on Genetic Algorithms (ICGA) and the 26th Annual Genetic Programming Conference (GP)",
