Routine high-return human-competitive automated problem-solving by means of genetic programming
Introduction
One of the central challenges of computer science is to get a computer to solve a problem without explicitly programming it to do so. Paraphrasing Arthur Samuel—founder of the field of machine learning—this challenge [20] concerns:
How can computers be made to do what needs to be done, without being told exactly how to do it?
In his 1983 talk entitled “AI: Where It Has Been and Where It Is Going,” Samuel [21] provided a criterion for success in achieving the goal by saying:
“[T]he aim [is]…to get machines to exhibit behavior, which if done by humans, would be assumed to involve the use of intelligence.”
Genetic programming starts from a high-level statement of what needs to be done and automatically creates a computer program to solve the problem. Genetic programming uses the Darwinian principle of natural selection and analogs of recombination (crossover), mutation, gene duplication, gene deletion, and certain mechanisms of developmental biology to progressively breed an improved population over a series of many generations.
This paper makes four points:
- (1)
Genetic programming (described briefly in Section 2) now routinely delivers high-return human-competitive machine intelligence (Section 3).
- (2)
Genetic programming is an automated invention machine (Section 4).
- (3)
Genetic programming can automatically create a general solution to a problem in the form of a parameterized topology (Section 5).
- (4)
Genetic programming has delivered a progression of qualitatively more substantial results in synchrony with five approximately order-of-magnitude increases in the expenditure of computer time (Section 6).
These points are illustrated by a group of recent results involving the automatic synthesis of:
- •
both the topology and sizing of analog electrical circuits (Section 7),
- •
placement and routing (i.e., layout) of circuits (performed automatically and simultaneously with the synthesis of the circuit’s topology and sizing) (Section 8),
- •
automatic synthesis of parameterized topologies for a general-purpose controller (Section 9),
- •
automatic synthesis of parameterized topologies containing conditional developmental operators (Section 10), and
- •
automatic synthesis of antennas, mathematical algorithms, classifiers of protein sequences, networks of chemical reactions (metabolic pathways), and genetic networks (Section 11).
Section snippets
Background on genetic programming
Genetic programming starts with a high-level description of “what needs to be done” and automatically executes an iterative procedure in an attempt to create a computer program that does what is required [9], [10], [11], [12], [13], [14], [15], [16].
The preparatory steps for a run of genetic programming are the problem-specific and domain-specific steps that are performed by the human user prior to launching a run. The executional steps are the problem-independent and domain-independent steps
Definition of “Human-Competitive”
In attempting to evaluate an automated problem-solving method, the question arises as to whether there is any real substance to the demonstrative problems that are published in connection with the method. Demonstrative problems in the fields of artificial intelligence and machine learning are often contrived toy problems that circulate exclusively inside academic groups that study a particular methodology. These problems typically have little relevance to any issues pursued by any scientist or
Genetic programming is an automated invention machine
There are now 23 instances where genetic programming has duplicated the functionality of a previously patented invention, infringed a previously issued patent, or created a patentable new invention. These 23 instances are indicated by an “A” in column 3 of Table 2. Specifically, there are 15 instances where genetic programming has created an entity that either infringes or duplicates the functionality of a previously patented 20th-century invention, six instances where genetic programming has
Genetic programming can automatically create parameterized topologies
Genetic programming can automatically create, in a single run, a general (parameterized) solution to a problem in the form of a graphical structure whose nodes or edges represent components and where the parameter values of the components are specified by mathematical expressions containing free variables. We call such a solution a parameterized topology.
In a parameterized topology, the genetically evolved graphical structure represents a complex structure (e.g., electrical circuit,
Progression of qualitatively more substantial results produced by genetic programming in synchrony with increasing computer power
Table 3 lists the five computer systems used to produce our group’s reported work on genetic programming in the 15-year period between 1987 and 2002. Column 7 shows the number of human-competitive results (as itemized in Table 2) generated by each computer system.
The first entry in the table is a serial computer. The four subsequent entries are parallel computer systems. The presence of four increasingly powerful parallel computer systems in the table reflects the fact that genetic programming
Automatic synthesis of the topology and sizing of patented analog circuits
Patents represent current research and development efforts of the engineering and scientific communities.
Genetic programming can automatically create both the topology (graphical structure) and sizing (numerical component values) for patented analog electrical circuits composed of transistors, capacitors, resistors, and other components. Genetic programming starts from a high-level statement of a circuit’s desired behavior and characteristics (e.g., its desired output given its input). It uses
Topology, sizing, placement, and routing of circuits
Circuit placement involves the assignment of each of the circuit’s components to a particular physical location on a printed circuit board or silicon wafer. Routing involves the assignment of a particular physical location to the wires between the leads of the circuit’s components.
Genetic programming can simultaneously create a circuit’s topology and sizing along with the placement and routing of all components as part of an integrated overall design process [13]. It can do all four of these
General-purpose controller problem
The purpose of a controller is to force, in a meritorious way, the response of a system (conventionally called the plant) to match a desired response (the reference signal). Controllers are ubiquitous in the real world. PID controllers are composed of a proportional (P), an integrative (I), and a derivative (D) block. Over 90% of all real-world controllers are PID controllers [1].
In 1942, Ziegler and Nichols developed a set of mathematical rules for automatically selecting the parameter values
Automatic synthesis of parameterized topologies containing conditional operators
If the genetically evolved program contains conditional developmental operators as well as free variables, a different graphical structure will, in general, be produced for different instantiations of the free variables. That is, the genetically evolved program operates as a genetic switch. Each program is provided with the problem’s free variables as input. These values trigger the development of different graphical structures as the program is executed.
For example, a single genetically
Other application areas
Genetic programming has also produced human-competitive results involving the automatic synthesis of antennas, mathematical algorithms, classifiers of protein sequences, and networks of chemical reactions. Antennas are automatically created using a “turtle” that deposits (or fails to deposit) metal on a plane as it moves and turns under the control of various operations similar to those found in the LOGO programming language or Lindenmayer systems [15]. The genetically evolved programs for
Conclusions
This paper demonstrated that genetic programming (1) now routinely delivers high-return human-competitive machine intelligence; (2) is an automated invention machine; (3) can automatically create a general solution to a problem in the form of a parameterized topology and (4) has delivered a progression of qualitatively more substantial results in synchrony with five approximately order-of-magnitude increases in the expenditure of computer time.
References (23)
- et al.
PID Controllers: Theory, Design, and Tuning
(1995) - T.S. Aytur, Integrated circuit with variable capacitor, US patent 6,013,958, Filed July 23, 1998, Issued January 11,...
- G.A. Campbell, Electric wave filter, US patent 1,227,113, Filed July 15, 1915, Issued May 22,...
- S. Darlington, Semiconductor signal translating device, US patent 2,663,806, Filed May 9, 1952, Issued December 22,...
- S. Cipriani, A.A. Takeshian, Compact cubic function generator, US patent 6,160,427, Filed September 4, 1998, Issued...
- T.C. Daun-Lindberg, M.L. Miller, Low voltage high-current electronic load, US patent 6,211,726, Filed June 28, 1999,...
- A. Ikeuchi, N. Tokuda, Voltage–current conversion circuit, US patent 6,166,529, Filed February 24, 2000, Issued...
- R. Irvine, B. Kolb, Integrated low-pass filter, US patent 6,225,859, Filed September 14, 1998, Issued May 1,...
Genetic Programming: On the Programming of Computers by Means of Natural Selection
(1992)- et al.
Genetic Programming: The Movie
(1992)
Genetic Programming II: Automatic Discovery of Reusable Programs
Cited by (25)
Stability assessment of rubble-mound breakwaters using genetic programming
2016, Ocean EngineeringA genetic programming-based QSPR model for predicting solubility parameters of polymers
2015, Chemometrics and Intelligent Laboratory SystemsCitation Excerpt :Mutation operator exchanges a randomly chosen part of one selected program in order to create one new offspring program for the new population. The best offspring program appearing in any generation, or the best-so-far solution, is able to solve the given problem in the best way [18]. In this study, the implementations were performed using computer programs that were written in MATLAB run on an Intel Core i7 based PC.
Prediction of the Unified Parkinson's Disease Rating Scale assessment using a genetic programming system with geometric semantic genetic operators
2014, Expert Systems with ApplicationsGenetic programming based blind image deconvolution for surveillancesystems
2013, Engineering Applications of Artificial IntelligenceCitation Excerpt :Crossover operator creates offspring by exchanging genetic material between two individual parents. To obtain good results through crossover, we used tournament selection method (Koza et al., 2008). This selection works by selecting trees at random from the current generation.
Repairing fractures between data using genetic programming-based feature extraction: A case study in cancer diagnosis
2013, Information SciencesCitation Excerpt :GP has been applied often to classification [13]. Among the latest advances in the field, we would like to mention those dedicated to high dimensional problems [35,6], variations in population size [31,32], and applications to other related fields [58,3]. Feature extraction creates new features as functional mappings of the old ones.
Evolving estimators of the pointwise Hölder exponent with Genetic Programming
2012, Information Sciences