Abstract
Many seemingly different problems in machine learning, artificial intelligence, and symbolic processing can be viewed as requiring the discovery of a computer program that produces some desired output for particular inputs. When viewed in this way, the process of solving these problems becomes equivalent to searching a space of possible computer programs for a highly fit individual computer program. The recently developed genetic programming paradigm described herein provides a way to search the space of possible computer programs for a highly fit individual computer program to solve (or approximately solve) a surprising variety of different problems from different fields. In genetic programming, populations of computer programs are genetically bred using the Darwinian principle of survival of the fittest and using a genetic crossover (sexual recombination) operator appropriate for genetically mating computer programs. Genetic programming is illustrated via an example of machine learning of the Boolean 11-multiplexer function and symbolic regression of the econometric exchange equation from noisy empirical data.
Hierarchical automatic function definition enables genetic programming to define potentially useful functions automatically and dynamically during a run, much as a human programmer writing a complex computer program creates subroutines (procedures, functions) to perform groups of steps which must be performed with different instantiations of the dummy variables (formal parameters) in more than one place in the main program. Hierarchical automatic function definition is illustrated via the machine learning of the Boolean 11-parity function.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Belew, R. and Booker, L. (eds) (1991). Proceedings of the Fourth International Conference on Genetic Algorithms. Morgan Kaufmann, San Mateo, CA.
Citibank (1989). CITIBASE: Citibank Economic Database (Machine Readable Magnetic Data File), 1946-Present. Citibank N.A., New York.
Cramer, N. L. (1985). A representation for the adaptive generation of simple sequential programs. In Proceedings of an International Conference on Genetic Algorithms and Their Applications, ed. J. Grefenstette. Lawrence Erlbaum, Hillsdale, NJ.
Davidor, Y. (1991). Genetic Algorithms and Robotics. World Scientific, Singapore.
Davis, L. (ed) (1987). Genetic Algorithms and Simulated Annealing. Pitman, London.
Davis, L. (1991). Handbook of Genetic Algorithms. Van Nostrand Reinhold, New York.
Forrest, S. (ed). (1990). Emergent Computation: Self-Organizing, Collective, and Cooperative Computing Networks. MIT Press, Cambridge, MA.
Fujiki, C. and Dickinson, J. (1987). Using the genetic algorithm to generate LISP source code to solve the prisoner's dilemma. In Genetic Algorithms and Their Applications: Proceedings of the Second International Conference on Genetic Algorithms, ed. J. Grefenstette. Lawrence Erlbaum, Hillsdale, NJ.
Goldberg, D. (1989). Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley, Reading, MA.
Goldberg, D. E., Korb, B. and Deb, K. (1989). Messy genetic algorithms: motivation, analysis, and first results. Complex Systems, 3, 493–530.
Gruau, F. (1992). Genetic synthesis of Boolean neural networks with a cell rewriting developmental process. In Proceedings of the Workshop on Combinations of Genetic Algorithms and Neural Networks 1992, ed. J. D. Schaffer and D. Whitley. The IEEE Computer Society Press.
Hallman, J. J., Porter, R. D. and Small, D. H. (1989). M2 per Unit of Potential GNP as an Anchor for the Price Level. Board of Governors of the Federal Reserve System. Staff Study 157, Washington, DC.
Holland, J. H. (1975). Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor, MI.
Holland, J. H. (1986). Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rulebased systems. In Machine Learning: An Artificial Intelligence Approach, Volume II, ed. R. S. Michalski et al. pp. 593–623. Morgan Kaufmann, Los Altos, CA.
Holland, J. H., Holyoak, K. J., Nisbett, R. E. and Thagard, P. A. (1986). Induction: Processes of Inference, Learning, and Discovery. MIT Press, Cambridge, MA.
Koza, J. R. (1990). Genetic Programming: A Paradigm for Genetically Breeding Populations of Computer Programs to Solve Problems. Stanford University Computer Science Department technical report STAN-CS-90-1314.
Koza, J. R. (1992a). Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA.
Koza, J. R. (1992b). Genetic programming: genetically breeding populations of computer programs to solve problems. In Dynamic, Genetic, and Chaotic Programming, ed. B. Soucek and the IRIS Group. John Wiley, New York.
Koza, J. R. (1992c). Hierarchical automatic function definition in genetic programming. In Proceedings of Workshop on the Foundations of Genetic Algorithms and Classifier Systems, Vail, Colorado 1992, ed. D. Whitley. Morgan Kaufmann, San Mateo, CA.
Koza, J. R. (1992d). Evolution of subsumption using genetic programming. In Proceedings of European Conference on Artificial Life, Paris, December 1991, ed. P. Bourgine and F. Varela. MIT Press, Cambridge, MA.
Koza, J. R. and Keane, M. A. (1990a). Cart centering and broom balancing by genetically breeding populations of control strategy programs. In Proceedings of International Joint Conference on Neural Networks, Washington, January 15–19,1990. Volume I, pp. 198–201. Lawrence Erlbaum, Hillsdale, NJ.
Koza, R. and Keane, M. A. (1990b). Genetic breeding of nonlinear optimal control strategies for broom balancing. In Proceedings of the Ninth International Conference on Analysis and Optimization of Systems. Antibes, France, June, 1990, pp. 47–56. Springer-Verlag, Berlin.
Koza, J. R. and Rice, J. P. (1991). Genetic generation of both the weights and architecture for a neural network. In Proceedings of International Joint Conference on Neural Networks, Seattle, July 1991. Volume II, pp. 397–404. IEEE Press.
Koza, J. R. and Rice, J. P. (1992a). Genetic Programming: The Movie. MIT Press, Cambridge, MA.
Koza, J. R. and Rice, J. P. (1992b). Automatic programming of robots using genetic programming. In Proceedings of Tenth National Conference on Artificial Intelligence, pp. 194–201. AAAI Press/MIT Press, Menlo Park, CA.
Koza, J. R., Keane, M. A. and Rice, J. P. (1993). Performance improvement of machine learning via automatic discovery of facilitating functions as applied to a problem of symbolic system identification. In 1993 IEEE International Conference on Neural Networks, San Francisco, Volume I, pp. 191–198. IEEE Press, Piscataway, NJ.
Langton, C., Taylor, C., Farmer, J. D. and Rasmussen, S. (eds). (1992). Artificial Life II, SFI Studies in the Sciences of Complexity, Volume X. Addison-Wesley, Redwood City, CA.
Meyer, J.-A. and Wilson, S. W. (1991). From Animals to Animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior, Paris. September 24–28, 1990. MIT Press, Cambridge, MA.
Michaelewicz, Z. (1992). Genetic Algorithms + Data Structures = Evolution Programs. Springer-Verlag, New York.
Rawlins, G. (ed) (1991). Proceedings of Workshop on the Foundations of Genetic Algorithms and Classifier Systems, Bloomington, Indiana, July 15–18, 1990. Morgan Kaufmann, San Mateo, CA
Samuel, A. L. (1959). Some studies in machine learning using the game of checkers. IBM Journal of Research and Development, 3, 210–229.
Schaffer, J. D. (ed) (1989). Proceedings of the Third International Conference on Genetic Algorithms. Morgan Kaufmann, San Mateo, CA.
Schwefel, H.-P. and Maenner, R. (eds) (1991). Parallel Problem Solving from Nature. Springer-Verlag, Berlin.
Smith, S. F. (1980). A Learning System Based on Genetic Adaptive Algorithms. PhD dissertation, University of Pittsburgh, Pittsburgh, PA.
Whitley, D. (ed) (1992). Proceedings of Workshop on the Foundations of Genetic Algorithms and Classifier Systems, Vail, Colorado 1992. Morgan Kaufmann, San Mateo, CA.
Wilson, S. W. (1987a). Classifier systems and the animat problem. Machine Learning, 3, 199–228.
Wilson, S. W. (1987b). Hierarchical credit allocation in a classifier system. In Proceedings of the Tenth International Joint Conference on Artificial Intelligence, pp. 217–220. Morgan Kaufmann, San Mateo, CA.
Wilson, S. W. (1988). Bid competition and specificity reconsidered. Jounal of Complex Systems, 2, 705–723.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Koza, J.R. Genetic programming as a means for programming computers by natural selection. Stat Comput 4, 87–112 (1994). https://doi.org/10.1007/BF00175355
Issue Date:
DOI: https://doi.org/10.1007/BF00175355