Skip to main content
SpringerLink
Log in

Multi-Objective Methods for Tree Size Control

  • Published:
Genetic Programming and Evolvable Machines Aims and scope Submit manuscript

Abstract

Variable length methods for evolutionary computation can lead to a progressive and mainly unnecessary growth of individuals, known as bloat. First, we propose to measure performance in genetic programming as a function of the number of nodes, rather than trees, that have been evaluated. Evolutionary Multi-Objective Optimization (EMOO) constitutes a principled way to optimize both size and fitness and may provide parameterless size control. Reportedly, its use can also lead to minimization of size at the expense of fitness. We replicate this problem, and an empirical analysis suggests that multi-objective size control particularly requires diversity maintenance. Experiments support this explanation.

The multi-objective approach is compared to genetic programming without size control on the 11-multiplexer, 6-parity, and a symbolic regression problem. On all three test problems, the method greatly reduces bloat and significantly improves fitness as a function of computational expense. Using the FOCUS algorithm, multi-objective size control is combined with active pursuit of diversity, and hypothesized minimum-size solutions to 3-, 4- and 5-parity are found. The solutions thus found are furthermore easily interpretable. When combined with diversity maintenance, EMOO can provide an adequate and parameterless approach to size control in variable length evolution.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. P. J. Angeline, “Genetic programming and emergent intelligence, ” in Advances in Genetic Programming, K. E. Kinnear Jr. (ed.), MIT Press: Cambridge, MA, 1994, pp. 75-98.

    Google Scholar 

  2. P. J. Angeline and J. B. Pollack, “The evolutionary induction of subroutines, ” in Proceedings of the Fourteenth Annual Conference of the Cognitive Science Society, Lawrence Erlbaum: Bloomington, Indiana, USA, 1992, pp. 236-241.

    Google Scholar 

  3. W. Banzhaf, D. Banscherus and P. Dittrich, “Hierarchical genetic programming using local modules, ” Interjounrnal Complex Systems (228), 1998, URL: http://www.interjournal.org.

  4. W. Banzhaf and W. B. Langdon, “Some considerations on the reason for bloat, ” Genetic Programming and Evolvable Machines, vol. 3,no. 1, pp. 81-91, 2002.

    Google Scholar 

  5. W. Banzhaf, P. Nordin, R. E. Keller, and F. D. Francone, Genetic Programming—An Introduction: On the Automatic Evolution of Computer Programs and its Applications, dpunkt—Verlag für digitale Technologie GbmH and Morgan Kaufmann Publishers Inc.: Heidelberg and San Francisco, CA, 1998.

    Google Scholar 

  6. T. Blickle and L. Thiele, “Genetic programming and redundancy, ” in Genetic Algorithms within the Framework of Evolutionary Computation. Workshop at KI-94, J. Hopf (ed.), Max-Planck-Institut für Informatik (MPI-I-94-241): Saarbrücken, Germany, 1994, pp. 33-38.

    Google Scholar 

  7. A. Brindle, Genetic Algorithms for Function Optimization. Unpublished doctoral dissertation, (Computer Science Department, Technical Report TR81-2), University of Alberta, Canada.

  8. C. A. C. Coello, “An updated survey of GA-Based multiobjective optimization techniques, ” ACM Computing Surveys, vol. 32,no. 2, pp. 109-143, 2000.

    Google Scholar 

  9. W. Conover, Practical Nonparametric Statistics, Wiley: New York, NY, 1980.

    Google Scholar 

  10. D. W. Corne, N. R. Jerram, J. D. Knowles, and M. J. Oates, “PESA-II: Region-based selection in evolutionary multiobjective optimization, ” in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO-2001, L. Spector, E. D. Goodman, A. Wu, W. B. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, and E. Burke (eds.), Morgan Kaufmann: San Francisco, CA, 2001, pp. 283-290.

    Google Scholar 

  11. N. L. Cramer, “A representation for the adaptive generation of simple sequential programs, ” in Proceedings of an International Conference on Genetic Algorithms and their Applications, J. J. Grefenstette (ed.), Pittsburgh, PA, 1985, pp. 183-187.

  12. E. D. De Jong, R. A. Watson, and J. B. Pollack, “Reducing bloat and promoting diversity using multi-objective methods, ” in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO-2001, L. Spector, E. Goodman, A. Wu, W. Langdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. Garzon, and E. Burke (eds.), Morgan Kaufmann Publishers: San Francisco, CA, 2001, pp. 11-18.

    Google Scholar 

  13. K. Deb, Multi-objective Optimization using Evolutionary Algorithms, Wiley: Chichester, UK, 2001.

    Google Scholar 

  14. K. Deb, S. Agrawal, A. Pratab, and T. Meyarivan, “A fast elitist non-dominated sorting genetic algorithm for multi-objective optimization: NSGA-II, ” in Parallel Problem Solving from Nature—PPSN VI (Vol. 1917), M. Schoenauer, K. Deb, G. Rudolph, X. Yao, E. Lutton, J. J. Merelo, and H.-P. Schwefel (eds.), Springer Verlag, 2000, pp. 849-858.

  15. A. Ekárt and S. Németh, “Selection based on the Pareto nondomination criterion for controlling code growth in genetic programming, ” Genetic Programming and Evolvable Machines, vol. 2, pp. 61-73, 2001.

    Google Scholar 

  16. C. M. Fonseca and P. J. Fleming, “Genetic algorithms for multiobjective optimization: formulation, discussion and generalization, ” in Proceedings of the Fifth International Conference on Genetic Algorithms, ICGA-93, S. Forrest (ed.), Morgan Kauffman Publishers: San Mateo, CA, 1993, pp. 416-423.

    Google Scholar 

  17. C. M. Fonseca and P. J. Fleming, “An overview of evolutionary algorithms in multiobjective optimization, ” Evolutionary Computation, vol. 3,no. 1, pp. 1-16, 1995.

    Google Scholar 

  18. M. P. Fourman, “Compaction of symbolic layout using genetic algorithms, ” in Proceedings of the First International Conference on Genetic Algorithms and their Applications, J. J. Grefenstette (ed.), Lawrence Erlbaum Associates: Hillsdale, New Jersey, 1985, pp. 141-153.

    Google Scholar 

  19. M. Keijzer, Scientific Discovery using Genetic Programming, Unpublished doctoral dissertation, Danish Technical University, Lyngby.

  20. K. E. Kinnear, Jr, “Generality and difficulty in genetic programming: Evolving a sort, ” in Proceedings of the 5th International Conference on Genetic Algorithms, ICGA-93, S. Forrest (ed.), Morgan Kaufmann: San Mateo, CA, 1993, pp. 287-294.

    Google Scholar 

  21. J. R. Koza, Genetic Programming, MIT Press: Cambridge, MA, 1992.

    Google Scholar 

  22. J. R. Koza, Genetic Programming II: Automatic Discovery of Reusable Programs, MIT Press: Cambridge, MA, 1994.

    Google Scholar 

  23. W. B. Langdon, “Data structures and genetic programming, ” in Advances in Genetic Programming 2, P. J. Angeline and K. Kinnear (eds.), MIT Press: Cambridge, MA, 1996, pp. 395-414.

    Google Scholar 

  24. W. B. Langdon, “The evolution of size in variable length representations, ” in 1998 IEEE International Conference on Evolutionary Computation, IEEE Press: Anchorage, Alaska, USA, 1998, pp. 633-638.

    Google Scholar 

  25. W. B. Langdon and J. P. Nordin, “Seeding GP populations, ” in Genetic Programming, Proceedings of EuroGP-2000 (Vol. 1802), R. Poli, W. Banzhaf, W. B. Langdon, J. F. Miller, P. Nordin, and T. C. Fogarty (eds.), Springer-Verlag: Edinburgh, 2000, pp. 304-315.

    Google Scholar 

  26. W. B. Langdon and R. Poli, “Fitness causes bloat: Mutation, ” in Proceedings of the First European Workshop on Genetic Programming (Vol. 1391), W. Banzhaf, R. Poli, M. Schoenauer, and T. C. Fogarty (eds.), Springer-Verlag: Paris, 1998, pp. 37-48.

    Google Scholar 

  27. W. B. Langdon and R. Poli, Foundations of Genetic Programming, Springer-Verlag, 2002.

  28. W. B. Langdon, T. Soule, R. Poli, and J. A. Foster, “The evolution of size and shape, ” in Advances in Genetic Programming 3, L. Spector, W. B. Langdon, U.-M. O'Reilly, P. J. Angeline (eds.), MIT Press: Cambridge, MA, 1999, pp. 163-190.

    Google Scholar 

  29. M. Laumanns, L. Thiele, K. Deb, and E. Zitzler, Combining convergence and diversity in evolutionary multi-objective optimization, Evolutionary Computation, vol. 10,no. 3, pp. 263-282, 2002.

    Google Scholar 

  30. S. Luke and L. Panait, “Fighting bloat with nonparametric parsimony pressure, ” in Parallel problem solving from nature—PPSN VII. (p. 411 ff.), H.-P. Schwefel, J.-J. Merelo Guervós, P. Adamidis, H.-G. Beyer, and J.-L. Fernández-Villacañas (eds.), Springer-Verlag: 2002.

  31. S. W. Mahfoud, Niching Methods for Genetic Algorithms, Unpublished doctoral dissertation, Urbana, IL, (IlliGAL Report 95001).

  32. N. F. McPhee and J. D. Miller, “Accurate replication in genetic programming, ” in Genetic Algorithms: Proceedings of the Sixth International Conference, ICGA-95, L. Eshelman (ed.), Morgan Kaufmann: Pittsburgh, PA, 1995, pp. 303-309.

    Google Scholar 

  33. P. Nordin and W. Banzhaf, “Complexity compression and evolution, ” in Genetic Algorithms: Proceedings of the Sixth International Conference, ICGA-95, L. Eshelman (ed.), Morgan Kaufmann: Pittsburgh, PA, 1995, pp. 310-317.

    Google Scholar 

  34. P. Nordin, F. Francone and W. Banzhaf, “Explicitly defined introns and destructive crossover in genetic programming, ” in Advances in Genetic Programming 2, P. J. Angeline and K. E. Kinnear Jr. (eds.), MIT Press: Cambridge, MA, 1996, pp. 111-134.

    Google Scholar 

  35. R. Olsson, “Inductive functional programming using incremental program transformation, ” Artificial Intelligence, vol. 74,no. 1, pp. 55-81, 1995.

    Google Scholar 

  36. R. Poli, “Discovery of symbolic, neuro-symbolic and neural networks with parallel distributed genetic programming. In 3rd International Conference on Artifical Neural Networks and Genetic Algorithms, ” in ICANNGA'97, Springer: Berlin, 1997.

    Google Scholar 

  37. R. Poli and N. F. McPhee, “Exact schema theorems for gp with one-point and standard crossover operating on linear structures and their application to the study of the evolution of size, ” in Genetic Programming. 4th European Conference, EuroGP 2001, J. Miller, M. Tomassini, P. Lanzi, C. Ryan, A. Tettamanzi, and W. Langdon (eds.), Springer: Berlin, 2001, pp. 126-142.

    Google Scholar 

  38. K. Rodríguez-Vázquez, C. M. Fonseca and P. J. Fleming, “Multi-objective genetic programming: A nonlinear system identification application, ” in Late Breaking Papers at the 1997 Genetic Programming Conference, J. R. Koza (ed.), Stanford Bookstore: Stanford University, CA, 1997, pp. 207-212.

    Google Scholar 

  39. J. Rosca, “Generality versus size in genetic programming, ” in Genetic Programming 1996: Proceedings of the First Annual Conference, J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo (eds.), MIT Press: Cambridge, MA, 1996, pp. 381-387.

    Google Scholar 

  40. J. D. Schaffer, “Multiple objective optimization with vector evaluated genetic algorithms, ” in Proceedings of the First International Conference on Genetic Algorithms and their Applications, J. J. Grefenstette (ed.), Lawrence Erlbaum Associates: Hillsdale, New Jersey, 1985, pp. 93-100.

    Google Scholar 

  41. S. Smith, A Learning System Based on Genetic Adaptive Algorithms, Unpublished doctoral dissertation, University of Pittsburgh.

  42. T. Soule, Code Growth in Genetic Programming, Unpublished doctoral dissertation, University of Idaho.

  43. T. Soule and J. A. Foster, “Effects of code growth and parsimony presure on populations in genetic programming, ” Evolutionary Computation, vol. 6,no. 4, pp. 293-309, 1999.

    Google Scholar 

  44. T. Soule and R. B. Heckendorn, “An analysis of the causes of code growth in genetic programming, ” Genetic Programming and Evolvable Machines, vol. 3, pp. 283-309, 2002.

    Google Scholar 

  45. B.-T. Zhang and H. Mühlenbein, “Balancing accuracy and parsimony in genetic programming, ” Evolutionary Computation, vol. 3,no. 1, pp. 17-38, 1995.

    Google Scholar 

  46. E. Zitzler and L. Thiele, “Multiobjective evolutionary algorithms: A comparative case study and the strength pareto approach, ” IEEE Transactions on Evolutionary Computation, vol. 3,no. 4, pp. 257-271, 1999.

    Google Scholar 

Download references

Author information

Author notes

  1. Edwin D. de Jong

    Present address: Institute of Information and Computing Sciences, Decision Support Systems Group, Utrecht University, The Netherlands

Authors and Affiliations

  1. Department of Computer Science, Brandeis University, Waltham, Massachusetts, 02454, USA

    Edwin D. de Jong & Jordan B. Pollack

Authors
  1. Edwin D. de Jong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jordan B. Pollack
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

de Jong, E.D., Pollack, J.B. Multi-Objective Methods for Tree Size Control. Genetic Programming and Evolvable Machines 4, 211–233 (2003). https://doi.org/10.1023/A:1025122906870

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1025122906870

Search

Navigation