- author = "Dominik Sobania",
- title = "On the Generalizability of Programs Synthesized by Grammar-Guided Genetic Programming",
- booktitle = "EuroGP 2021: Proceedings of the 24th European Conference on Genetic Programming",
- year = "2021",
- editor = "Ting Hu and Nuno Lourenco and Eric Medvet",
- series = "LNCS",
- volume = "12691",
- publisher = "Springer Verlag",
- address = "Virtual Event",
- pages = "130--145",
- month = "7-9 " # apr,
- organisation = "EvoStar, Species",
- keywords = "genetic algorithms, genetic programming, SBSE, Program synthesis, Lexicase selection, Generalizability, Software engineering",
- isbn13 = "978-3-030-72811-3",
-
DOI = "
doi:10.1007/978-3-030-72812-0_9",
- abstract = "Grammar-guided Genetic Programming is a common approach for program synthesis where the user's intent is given by a set of input/output examples. For use in real-world software development, the generated programs must work on previously unseen test cases too. Therefore, we study in this work the generalisability of programs synthesised by grammar-guided GP with lexicase selection. As benchmark, we analyze proportionate and tournament selection too. We find that especially for program synthesis problems with a low output cardinality (e.g., a Boolean output) lexicase selection overfits the training cases and does not generalize well to unseen test cases. An analysis using common software metrics shows for such a problem that lexicase selection generates more complex programs with many code lines and a heavier use of control structures compared to the other studied selection methods. Nevertheless, the generalizability can be improved when we do not stop a GP run as usual after a first program is found that solves all training cases correctly, but give GP more time to find further solution candidates (also solving correctly all training cases) and select the smallest program (measured with different software metrics) out of these.",
-
notes = "STGP, lines of code, McCabe 1976 cyclomatic
complexity, AST depth, AST node count. Seven problems
from \cite{Helmuth:2015:GECCO} grammars based on
PonyGE2. pop=3000, gens=100. Not standard GA, tree
based. Lexicase selection => but sometimes very low
generalisation. Proportionate and tournament selection
too. IF not
generalising.
https://gitlab.rlp.net/dsobania/progsys-grammars
http://www.evostar.org/2021/eurogp/ Part of \cite{Hu:2021:GP} EuroGP'2021 held in conjunction with EvoCOP2021, EvoMusArt2021 and EvoApplications2021",
