Applying genetic programming to PSB2: the next generation program synthesis benchmark suite
Created by W.Langdon from
gp-bibliography.bib Revision:1.6704
- @Article{Helmuth:2022:GPEM,
-
author = "Thomas Helmuth and Peter Kelly",
-
title = "Applying genetic programming to {PSB2}: the next
generation program synthesis benchmark suite",
-
journal = "Genetic Programming and Evolvable Machines",
-
note = "Special Issue: Highlights of Genetic Programming 2021
Events",
-
note = "Online first",
-
keywords = "genetic algorithms, genetic programming, Automatic
program synthesis, Benchmarking",
-
ISSN = "1389-2576",
-
DOI = "
doi:10.1007/s10710-022-09434-y",
-
abstract = "For the past seven years, researchers in genetic
programming and other program synthesis disciplines
have used the General Program Synthesis Benchmark Suite
(PSB1) to benchmark many aspects of systems that
conduct programming by example, where the
specifications of the desired program are given as
input/output pairs. PSB1 has been used to make notable
progress toward the goal of general program synthesis:
automatically creating the types of software that human
programmers code. Many of the systems that have
attempted the problems in PSB1 have used it to
demonstrate performance improvements granted through
new techniques. Over time, the suite has gradually
become outdated, hindering the accurate measurement of
further improvements. The field needs a new set of more
difficult benchmark problems to move beyond what was
previously possible and ensure that systems do not
overfit to one benchmark suite. In this paper, we
describe the 25 new general program synthesis benchmark
problems that make up PSB2, a new benchmark suite.
These problems are curated from a variety of sources,
including programming katas and college courses. We
selected these problems to be more difficult than those
in the original suite, and give results using PushGP
showing this increase in difficulty. We additionally
give an example of benchmarking using a
state-of-the-art parent selection method, showing
improved performance on PSB2 while still leaving plenty
of room for improvement. These new problems will help
guide program synthesis research for years to come.",
- }
Genetic Programming entries for
Thomas Helmuth
Peter Kelly
Citations