Elsevier

Information Sciences

Volume 178, Issue 23, 1 December 2008, Pages 4434-4452
Information Sciences

Routine high-return human-competitive automated problem-solving by means of genetic programming

https://doi.org/10.1016/j.ins.2008.07.028Get rights and content

Abstract

Genetic programming is a systematic method for getting computers to automatically solve problems. Genetic programming starts from a high-level statement of what needs to be done and automatically creates a computer program to solve the problem by means of a simulated evolutionary process. The paper demonstrates that genetic programming (1) now routinely delivers high-return human-competitive machine intelligence; (2) is an automated invention machine; (3) can automatically create a general solution to a problem in the form of a parameterized topology and (4) has delivered a progression of qualitatively more substantial results in synchrony with five approximately order-of-magnitude increases in the expenditure of computer time. These points are illustrated by a group of recent results involving the automatic synthesis of the topology and sizing of analog electrical circuits, the automatic synthesis of placement and routing of circuits, and the automatic synthesis of controllers as well as references to work involving the automatic synthesis of antennas, networks of chemical reactions (metabolic pathways), genetic networks, mathematical algorithms, and protein classifiers.

Introduction

One of the central challenges of computer science is to get a computer to solve a problem without explicitly programming it to do so. Paraphrasing Arthur Samuel—founder of the field of machine learning—this challenge [20] concerns:

How can computers be made to do what needs to be done, without being told exactly how to do it?

In his 1983 talk entitled “AI: Where It Has Been and Where It Is Going,” Samuel [21] provided a criterion for success in achieving the goal by saying:

“[T]he aim [is]…to get machines to exhibit behavior, which if done by humans, would be assumed to involve the use of intelligence.”

Genetic programming starts from a high-level statement of what needs to be done and automatically creates a computer program to solve the problem. Genetic programming uses the Darwinian principle of natural selection and analogs of recombination (crossover), mutation, gene duplication, gene deletion, and certain mechanisms of developmental biology to progressively breed an improved population over a series of many generations.

This paper makes four points:

  • (1)

    Genetic programming (described briefly in Section 2) now routinely delivers high-return human-competitive machine intelligence (Section 3).

  • (2)

    Genetic programming is an automated invention machine (Section 4).

  • (3)

    Genetic programming can automatically create a general solution to a problem in the form of a parameterized topology (Section 5).

  • (4)

    Genetic programming has delivered a progression of qualitatively more substantial results in synchrony with five approximately order-of-magnitude increases in the expenditure of computer time (Section 6).

These points are illustrated by a group of recent results involving the automatic synthesis of:

  • both the topology and sizing of analog electrical circuits (Section 7),

  • placement and routing (i.e., layout) of circuits (performed automatically and simultaneously with the synthesis of the circuit’s topology and sizing) (Section 8),

  • automatic synthesis of parameterized topologies for a general-purpose controller (Section 9),

  • automatic synthesis of parameterized topologies containing conditional developmental operators (Section 10), and

  • automatic synthesis of antennas, mathematical algorithms, classifiers of protein sequences, networks of chemical reactions (metabolic pathways), and genetic networks (Section 11).

Section snippets

Background on genetic programming

Genetic programming starts with a high-level description of “what needs to be done” and automatically executes an iterative procedure in an attempt to create a computer program that does what is required [9], [10], [11], [12], [13], [14], [15], [16].

The preparatory steps for a run of genetic programming are the problem-specific and domain-specific steps that are performed by the human user prior to launching a run. The executional steps are the problem-independent and domain-independent steps

Definition of “Human-Competitive”

In attempting to evaluate an automated problem-solving method, the question arises as to whether there is any real substance to the demonstrative problems that are published in connection with the method. Demonstrative problems in the fields of artificial intelligence and machine learning are often contrived toy problems that circulate exclusively inside academic groups that study a particular methodology. These problems typically have little relevance to any issues pursued by any scientist or

Genetic programming is an automated invention machine

There are now 23 instances where genetic programming has duplicated the functionality of a previously patented invention, infringed a previously issued patent, or created a patentable new invention. These 23 instances are indicated by an “A” in column 3 of Table 2. Specifically, there are 15 instances where genetic programming has created an entity that either infringes or duplicates the functionality of a previously patented 20th-century invention, six instances where genetic programming has

Genetic programming can automatically create parameterized topologies

Genetic programming can automatically create, in a single run, a general (parameterized) solution to a problem in the form of a graphical structure whose nodes or edges represent components and where the parameter values of the components are specified by mathematical expressions containing free variables. We call such a solution a parameterized topology.

In a parameterized topology, the genetically evolved graphical structure represents a complex structure (e.g., electrical circuit,

Progression of qualitatively more substantial results produced by genetic programming in synchrony with increasing computer power

Table 3 lists the five computer systems used to produce our group’s reported work on genetic programming in the 15-year period between 1987 and 2002. Column 7 shows the number of human-competitive results (as itemized in Table 2) generated by each computer system.

The first entry in the table is a serial computer. The four subsequent entries are parallel computer systems. The presence of four increasingly powerful parallel computer systems in the table reflects the fact that genetic programming

Automatic synthesis of the topology and sizing of patented analog circuits

Patents represent current research and development efforts of the engineering and scientific communities.

Genetic programming can automatically create both the topology (graphical structure) and sizing (numerical component values) for patented analog electrical circuits composed of transistors, capacitors, resistors, and other components. Genetic programming starts from a high-level statement of a circuit’s desired behavior and characteristics (e.g., its desired output given its input). It uses

Topology, sizing, placement, and routing of circuits

Circuit placement involves the assignment of each of the circuit’s components to a particular physical location on a printed circuit board or silicon wafer. Routing involves the assignment of a particular physical location to the wires between the leads of the circuit’s components.

Genetic programming can simultaneously create a circuit’s topology and sizing along with the placement and routing of all components as part of an integrated overall design process [13]. It can do all four of these

General-purpose controller problem

The purpose of a controller is to force, in a meritorious way, the response of a system (conventionally called the plant) to match a desired response (the reference signal). Controllers are ubiquitous in the real world. PID controllers are composed of a proportional (P), an integrative (I), and a derivative (D) block. Over 90% of all real-world controllers are PID controllers [1].

In 1942, Ziegler and Nichols developed a set of mathematical rules for automatically selecting the parameter values

Automatic synthesis of parameterized topologies containing conditional operators

If the genetically evolved program contains conditional developmental operators as well as free variables, a different graphical structure will, in general, be produced for different instantiations of the free variables. That is, the genetically evolved program operates as a genetic switch. Each program is provided with the problem’s free variables as input. These values trigger the development of different graphical structures as the program is executed.

For example, a single genetically

Other application areas

Genetic programming has also produced human-competitive results involving the automatic synthesis of antennas, mathematical algorithms, classifiers of protein sequences, and networks of chemical reactions. Antennas are automatically created using a “turtle” that deposits (or fails to deposit) metal on a plane as it moves and turns under the control of various operations similar to those found in the LOGO programming language or Lindenmayer systems [15]. The genetically evolved programs for

Conclusions

This paper demonstrated that genetic programming (1) now routinely delivers high-return human-competitive machine intelligence; (2) is an automated invention machine; (3) can automatically create a general solution to a problem in the form of a parameterized topology and (4) has delivered a progression of qualitatively more substantial results in synchrony with five approximately order-of-magnitude increases in the expenditure of computer time.

References (23)

  • K.J. Åström et al.

    PID Controllers: Theory, Design, and Tuning

    (1995)
  • T.S. Aytur, Integrated circuit with variable capacitor, US patent 6,013,958, Filed July 23, 1998, Issued January 11,...
  • G.A. Campbell, Electric wave filter, US patent 1,227,113, Filed July 15, 1915, Issued May 22,...
  • S. Darlington, Semiconductor signal translating device, US patent 2,663,806, Filed May 9, 1952, Issued December 22,...
  • S. Cipriani, A.A. Takeshian, Compact cubic function generator, US patent 6,160,427, Filed September 4, 1998, Issued...
  • T.C. Daun-Lindberg, M.L. Miller, Low voltage high-current electronic load, US patent 6,211,726, Filed June 28, 1999,...
  • A. Ikeuchi, N. Tokuda, Voltage–current conversion circuit, US patent 6,166,529, Filed February 24, 2000, Issued...
  • R. Irvine, B. Kolb, Integrated low-pass filter, US patent 6,225,859, Filed September 14, 1998, Issued May 1,...
  • J.R. Koza

    Genetic Programming: On the Programming of Computers by Means of Natural Selection

    (1992)
  • J.R. Koza et al.

    Genetic Programming: The Movie

    (1992)
  • J.R. Koza

    Genetic Programming II: Automatic Discovery of Reusable Programs

    (1994)
  • Cited by (25)

    • A genetic programming-based QSPR model for predicting solubility parameters of polymers

      2015, Chemometrics and Intelligent Laboratory Systems
      Citation Excerpt :

      Mutation operator exchanges a randomly chosen part of one selected program in order to create one new offspring program for the new population. The best offspring program appearing in any generation, or the best-so-far solution, is able to solve the given problem in the best way [18]. In this study, the implementations were performed using computer programs that were written in MATLAB run on an Intel Core i7 based PC.

    • Genetic programming based blind image deconvolution for surveillancesystems

      2013, Engineering Applications of Artificial Intelligence
      Citation Excerpt :

      Crossover operator creates offspring by exchanging genetic material between two individual parents. To obtain good results through crossover, we used tournament selection method (Koza et al., 2008). This selection works by selecting trees at random from the current generation.

    • Repairing fractures between data using genetic programming-based feature extraction: A case study in cancer diagnosis

      2013, Information Sciences
      Citation Excerpt :

      GP has been applied often to classification [13]. Among the latest advances in the field, we would like to mention those dedicated to high dimensional problems [35,6], variations in population size [31,32], and applications to other related fields [58,3]. Feature extraction creates new features as functional mappings of the old ones.

    View all citing articles on Scopus
    View full text