- author = "Dirk Schweim and Dominik Sobania and Franz Rothlauf",
- title = "Effects of the Training Set Size: A Comparison of Standard and Down-Sampled Lexicase Selection in Program Synthesis",
- booktitle = "2022 IEEE Congress on Evolutionary Computation (CEC)",
- year = "2022",
- editor = "Carlos A. Coello Coello and Sanaz Mostaghim",
- address = "Padua, Italy",
- month = "18-23 " # jul,
- keywords = "genetic algorithms, genetic programming, grammatical evolution, PonyGE2, RVD, grammar-guided GP, BNF, source code representation, Python, Down-Sampled Lexicase Selection, Training, Training data, abstract syntax tree, AST software metrics, LOC, cyclomatic complexity, AST depth, AST size, Lexical diversity, Evolutionary computation, Benchmark testing, Standards, Program synthesis, automatic programming, evolutionary algorithms",
- isbn13 = "978-1-6654-6708-7",
-
DOI = "
doi:10.1109/CEC55065.2022.9870337",
- size = "7 pages",
- abstract = "From a practitioners perspective, the number of input/output examples used during the training process in program synthesis studies is too large, as in practice, these examples must be labeled by hand. Therefore, this paper analyzes the influence of different training set sizes on the performance, generalization ability, as well as the structure of the programs generated by grammar-guided genetic programming. We compare down-sampled lexicase selection with standard lexicase selection on three common problems from the general program synthesis benchmark suite. First, we find that both lexicase variants are robust against reducing the amount of training data. We find that standard lexicase has a tendency to overfit the training data on some problems. With down sampled lexicase, in contrast, overfitting on training data is reduced and evolved programs generalize better on held-out test cases. Consequently, we suggest to use grammar-guided genetic programming with downsampled lexicase selection in the program synthesis domain.",
-
notes = "Also known as \cite{9870337}
'down-sampled lexicase selection often evolves smaller programs' cites \cite{Sobania:ieeeTEC} Compare String Length, Median, Smallest.
'In our grammar-guided GP, the initial population is generated using a GE initialization scheme. Then, crossover and mutation operators are applied to parse trees as in GP, but are only allowed to produce programs that obey the rules of the BNF grammar.'
'Lexical diversity is ... number of different node [types] in the AST' divided by the number of nodes. pop=1000, 300 generations.
'down-sampling lexicase is a good choice with grammar-guided GP' [PonyGE]. In future 'choose a much higher generation limit'.
",
