On the Difficulty of Benchmarking Inductive Program Synthesis Methods
Created by W.Langdon from
gp-bibliography.bib Revision:1.8081
- @InProceedings{Pantridge:2017:GECCOa,
-
author = "Edward Pantridge and Thomas Helmuth and
Nicholas Freitag McPhee and Lee Spector",
-
title = "On the Difficulty of Benchmarking Inductive Program
Synthesis Methods",
-
booktitle = "Proceedings of the Genetic and Evolutionary
Computation Conference Companion",
-
series = "GECCO '17",
-
year = "2017",
-
isbn13 = "978-1-4503-4939-0",
-
address = "Berlin, Germany",
-
pages = "1589--1596",
-
size = "8 pages",
-
URL = "http://doi.acm.org/10.1145/3067695.3082533",
-
DOI = "doi:10.1145/3067695.3082533",
-
acmid = "3082533",
-
publisher = "ACM",
-
publisher_address = "New York, NY, USA",
-
keywords = "genetic algorithms, genetic programming, benchmarking,
inductive program synthesis, machine learning",
-
month = "15-19 " # jul,
-
abstract = "A variety of inductive program synthesis (IPS)
techniques have recently been developed, emerging from
different areas of computer science. However, these
techniques have not been adequately compared on general
program synthesis problems. In this paper we compare
several methods on problems requiring solution programs
to handle various data types, control structures, and
numbers of outputs. The problem set also spans levels
of abstraction; some would ordinarily be approached
using machine code or assembly language, while others
would ordinarily be approached using high-level
languages. The presented comparisons are focused on the
possibility of success; that is, on whether the system
can produce a program that passes all tests, for all
training and unseen testing inputs. The compared
systems are Flash Fill, MagicHaskeller, TerpreT, and
two forms of genetic programming. The two genetic
programming methods chosen were PushGP and Grammar
Guided Genetic Programming. The results suggest that
PushGP and, to an extent, TerpreT and Grammar Guided
Genetic Programming are more capable of finding
solutions than the others, albeit at a higher
computational cost. A more salient observation is the
difficulty of comparing these methods due to
drastically different intended applications, despite
the common goal of program synthesis.",
-
notes = "Also known as
\cite{Pantridge:2017:DBI:3067695.3082533} GECCO-2017 A
Recombination of the 26th International Conference on
Genetic Algorithms (ICGA-2017) and the 22nd Annual
Genetic Programming Conference (GP-2017)",
- }
Genetic Programming entries for
Edward R Pantridge
Thomas Helmuth
Nicholas Freitag McPhee
Lee Spector
Citations