Abstract
By synthesizing a growing body ofwork in search processes that are not driven by explicit objectives, this paper advances the hypothesis that there is a fundamental problem with the dominant paradigm of objective-based search in evolutionary computation and genetic programming: Most ambitious objectives do not illuminate a path to themselves. That is, the gradient of improvement induced by ambitious objectives tends to lead not to the objective itself but instead to deadend local optima. Indirectly supporting this hypothesis, great discoveries often are not the result of objective-driven search. For example, the major inspiration for both evolutionary computation and genetic programming, natural evolution, innovates through an open-ended process that lacks a final objective. Similarly, large-scale cultural evolutionary processes, such as the evolution of technology, mathematics, and art, lack a unified fixed goal. In addition, direct evidence for this hypothesis is presented from a recently-introduced search algorithm called novelty search. Though ignorant of the ultimate objective of search, in many instances novelty search has counter-intuitively outperformed searching directly for the objective, including a wide variety of randomly-generated problems introduced in an experiment in this chapter. Thus a new understanding is beginning to emerge that suggests that searching for a fixed objective, which is the reigning paradigm in evolutionary computation and even machine learning as a whole, may ultimately limit what can be achieved. Yet the liberating implication of this hypothesis argued in this paper is that by embracing search processes that are not driven by explicit objectives, the breadth and depth of what is reachable through evolutionary methods such as genetic programming may be greatly expanded.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Barnett, Lionel (2001). Netcrawling - optimal evolutionary search with neutral networks. In Proc. of the 2001 IEEE Intl. Conf. on Evol. Comp., pages 30–37. IEEE Press.
Brockhoff, Dimo, Friedrich, Tobias, Hebbinghaus, Nils, Klein, Christian, Neumann, Frank, and Zitzler, Eckart (2007). Do additional objectives make a problem harder? In GECCO ’07: Proc. of the 9th Annual Conf. on Genetic and Evol. Comp., pages 765–772, New York, NY, USA. ACM.
Cliff, Dave andMiller, Geoffrey (1995). Tracking the red queen:Measurements of adaptive progress in co-evolutionary simulations. Advances in Artificial Life, pages 200–218.
Deb, Kalyanmoy (1999). Multi-objective genetic algorithms: Problem difficulties and construction of test problems. Evol. Comp., 7:205–230.
Doucette, John (2010).Novelty-based fitnessmeasures in genetic programming. Master of science in computer science, Dalhouise University.
Drexler, K.E. and Minsky, M. (1986). Engines of creation. Anchor Press.
Ficici, Sevan and Pollack, Jordan B. (1998). Challenges in coevolutionary learning: Arms-race dynamics, open-endedness, and mediocre stable states. In Proc. of the Sixth Intl. Conf. on Art. Life, pages 238–247. MIT Press.
Goldberg, David E. (1987). Simple genetic algorithms and the minimal deceptive problem. In Davis, L. D., editor, Genetic Algorithms and SimulatedAnnealing, Re- search Notes in Artificial Intelligence. Morgan Kaufmann.
Goldsby, H.J. and Cheng, B.H.C. (2010). Automatically Discovering Properties that Specify the Latent Behavior of UML Models. In Proceedings of MODELS 2010.
Gomez, Faustino and Miikkulainen, Risto (1997). Incremental evolution of complex general behavior. Adaptive Behavior, 5:317–342.
Gould, Steven Jay (1996). Full House: The Spread of Excellence from Plato to Darwin. Harmony Books.
Holland, John H. (1975). Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control and Artificial Intelligence. University of Michigan Press, Ann Arbor, MI.
Kelly, K. (2010). What technology wants. Viking Press.
Koza, John R., Keane, Martin A., Streeter, Matthew J., Mydlowec, William, Yu, Jessen, and Lanza, Guido (2003). Genetic Programming IV: Routine Human-Competitive Machine Intelligence. Kluwer Academic Publishers.
Lehman, Joel and Stanley, Kenneth O. (2008). Exploiting open-endedness to solve problems through the search for novelty. In Proc. of the Eleventh Intl. Conf. on Artificial Life (ALIFE XI), Cambridge, MA. MIT Press.
Lehman, Joel and Stanley, Kenneth O. (2010a). Efficiently evolving programs through the search for novelty. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2010). ACM.
Lehman, Joel and Stanley, Kenneth O. (2010b). Revising the evolutionary computation
abstraction: Minimal criteria novelty search. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2010). ACM.
Lehman, Joel and Stanley, Kenneth O. (2011). Abandoning objectives: Evolution through the search for novelty alone. Evol. Comp., 19(2):189–223.
Lynch, Michael (2007). The frailty of adaptive hypotheses for the origins of organismal complexity. In Proc Natl Acad SciUSA, volume104, pages 8597– 8604.
Mahfoud, SamirW. (1995). Nichingmethods for genetic algorithms. PhD thesis, University of Illinois at Urbana-Champaign, Champaign, IL, USA.
Mill, John Stuart (1846). A System of Logic, Ratiocinative and Inductive. John W. Parker and Son.
Mouret, Jean-Baptiste (2009). Novelty-based multiobjectivization. In Proc. of the Workshop on Exploring New Horizons in Evol. Design of Robots,2009 IEEE/RSJ Intl. Conf. on Intelligent Robots and Systems.
Pelikan, Martin, Pelikan, Martin, Goldberg, David E., and Goldberg, David E. (2001). Escaping hierarchical traps with competent genetic algorithms. In Proc. of the Genetic and Evolutionary Computation Conference (GECCO- 2001), pages 511–518. Morgan Kaufmann.
Reil, Torsten and Husbands, Phil (2002). Evolution of central pattern generators for bipedal walking in a real-time physics environment. IEEE Transactions on Evolutionary Computation, 6(2):159–168.
Reynolds, AM(2010). Maze-solving by chemotaxis. Physical Review E, 81(6).
Risi, S., Hughes, C.E., and Stanley, K.O. (2010). Evolving plastic neural networks with novelty search. Adaptive Behavior.
Schmidhuber, J. (2006). Developmental robotics, optimal artificial curiosity, creativity, music, and the fine arts. Connection Science, 18(2):173–187.
Secretan, J.,Beato,N.,D’Ambrosio,D.B.,Rodriguez,A.,Campbell,A., Folsom-
Kovarik, J.T., and Stanley, K.O. (2011). Picbreeder: A case study in collaborative
evolutionary exploration of design space. Evol. Comp. To appear.
Spector,Lee,Barnum,Howard,Bernstein,Herbert J., and Swamy,Nikhil (1999). Quantum computing applications of genetic programming. In Spector, Lee, Langdon, William B., O’Reilly, Una-May, and Angeline, Peter J., editors, Advances in Genetic Programming 3, chapter 7, pages 135–160. MIT Press, Cambridge, MA, USA.
Stanley, Kenneth O. andMiikkulainen, Risto (2002). Evolving neural networks through augmenting topologies. Evolutionary Computation, 10:99–127.
Stewart, T. C. (2001). Extrema selection: Accelerated evolution on neutral networks. In Proc. of the 2001 IEEE Intl. Conf. on Evol. Comp. IEEE Press.
Veldhuizen, David A. Van and Lamont, Gary B. (2000). Multiobjective evolutionary algorithms: Analyzing the state-of-the-art. Evolutionary Computation, 8(2):125–147.
Weinberger, Edward (1990). Correlated and uncorrelated fitness landscapes and how to tell the difference. Biological Cybernetics, 63(5):325–336.
Wolpert, David H. and Macready, William (1995). No free lunch theorems for search. Technical Report SFI-TR-95-01-010, The Santa Fe Institute, Santa Fe, NM.
Woolley, Brian G. and Stanley, Kenneth O. (2011). On the deleterious effects
of a priori objectives on evolution and representation. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2011). ACM. Yao, Xin (1999). Evolving artificial neural networks. Proceedings of the IEEE, 87(9):1423–1447.
Zaera, N., Cliff, D., and Bruten, J. (1996). (Not) evolving collective behaviours in synthetic fish. In From Animals to Animats 4: Proc. of the Fourth Intl. Conf. on Simulation of Adaptive Behavior. MIT Press Bradford Books.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Lehman, J., Stanley, K.O. (2011). Novelty Search and the Problem with Objectives. In: Riolo, R., Vladislavleva, E., Moore, J. (eds) Genetic Programming Theory and Practice IX. Genetic and Evolutionary Computation. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-1770-5_3
Download citation
DOI: https://doi.org/10.1007/978-1-4614-1770-5_3
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-1769-9
Online ISBN: 978-1-4614-1770-5
eBook Packages: Computer ScienceComputer Science (R0)