Skip to main content
SpringerLink
Log in

Prudent alignment and crossover of decision trees in genetic programming

  • Published:
Genetic Programming and Evolvable Machines Aims and scope Submit manuscript

Abstract

Crossover is the central search operator responsible for navigating through unknown problem landscapes while at the same time the main conservation operator, which is supposed to preserve the already learned lessons. This paper is about a novel homologous decision tree crossover operator. Contrary to other tree crossover operators it defines the context for a decision tree node and elaborates a fast one-sample-based tree alignment procedure. The idea is to replace a sub-tree with a better one from the same context, as defined by the decision tree training process. This operator does not rely on the topological properties of the tree but rather on its behavioral properties. During empirical testing the new operator showed the best generalization capabilities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. The first instance through this test will always travel to the false branch because \(res<res\) is never true.

  2. Other possibilities exist and are worth considering, but we did not explore them in this paper.

  3. Determined by the \(\beta \) parameter.

  4. Defining ‘better’ is the most problematic part of a fitness function.

  5. Only the standard crossover is documented well-enough to warrant correctness of implementation.

References

  1. W. Banzhaf, P. Nordin, R. Keller, F. Francone, Genetic Programming—An Introduction (Morgan Kaufmann, San Francisco, 1998)

    Book  MATH  Google Scholar 

  2. R. Barros, M. Basgalupp, A. de Carvalho, A. Freitas, A survey of evolutionary algorithms for decision-tree induction. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev. 42(3), 291–311 (2012)

    Article  Google Scholar 

  3. K. Bache, M. Lichman, UCI Machine Learning Repository (School of Information and Computer Sciences, University of California, Irvine, 2013). http://archive.ics.uci.edu/ml

  4. J. Bongard, A probabilistic functional crossover operator for genetic programming, in Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, GECCO ’10 (ACM, New York, NY, 2010), pp. 925–932

  5. L. Booker, D. Goldberg, J. Holland, Classifier systems and genetic algorithms. Artif. Intell. 40(1–3), 235–282 (1989)

    Article  Google Scholar 

  6. L. Breiman, J. Friedman, R. Olshen, C. Stone, Classification and Regression Trees (Wadsworth, Monterrey, CA, 1984)

    MATH  Google Scholar 

  7. S.H. Cha, C. Tappert, A genetic algorithm for constructing compact binary decision trees. J. Pattern Recognit. Res. 4(1), 1–13 (2009)

    Article  Google Scholar 

  8. M. Defoin-Platel, M. Clergue, P. Collard, Maximum homologous crossover for linear genetic programming, in Genetic Programming, 6th European Conference, EuroGP 2003, Lecture Notes in Computer Science, vol. 2610, ed. by C. Ryan, T. Soule, M. Keijzer, E. Tsang, R. Poli, E. Costa (Springer, Berlin, 2003), pp. 194–203

  9. K. DeJong, The Analysis of the Behaviour of a Class of Genetic adaptive Systems. Ph.D. thesis, (Department of Computer Science, University of Michigan, Ann Arbor, Michigan, 1975)

  10. L. Devroye, A note on the height of binary search trees. J. ACM 33(3), 489–498 (1986)

    Article  MATH  MathSciNet  Google Scholar 

  11. P. D’haeseleer, Context preserving crossover in genetic programming, in Proceedings of the 1994 IEEE World Congress on Computational Intelligence, vol. 1 (IEEE Press, New York, 1994), pp. 256–261

  12. J. Hernandez, B. Duval, J.K. Hao, A study of crossover operators for gene selection of microarray data, in Artificial Evolution, Lecture Notes in Computer Science, vol. 4926 (2008), pp. 243–254

  13. H. Kennedy, C. Chinniah, P. Bradbeer, L. Morss, The construction and evaluation of decision trees: a comparison of evolutionary and concept learning methods, in Evolutionary Computing, AISB Workshop, Lecture Notes in Computer Science, vol. 1305, ed. by D. Corne, J. Shapiro (Springer, Berlin, 1997), pp. 147–162

  14. J. Koza, Genetic Programming: On the Programming of Computers by Natural Selection (MIT Press, Cambridge, MA, 1992)

    MATH  Google Scholar 

  15. W.B. Langdon, Size fair and homologous tree crossovers for tree genetic programming, in Genetic Programming and Evolvable Machines, vol. 1 (Kluwer, Boston, 2000), pp. 95–119

  16. S. Luke, L. Panait, Is the perfect the enemy of the good, in Genetic and Evolutionary Computation Conference (Morgan Kaufmann, Los Altos, CA, 2002), pp. 820–828

  17. R. MacCallum, Introducing a perl genetic programming system—and can meta-evolution solve the bloat problem? in Genetic Programming, Proceedings of EuroGP, Lecture Notes in Computer Science, vol. 2610 (Springer, Berlin, 2003), pp. 369–378

  18. A. Moraglio, Towards a Geometric Unification of Evolutionary Algorithms. Ph.D. thesis (Department of Computer Science, University of Essex, 2007)

  19. A. Moraglio, One-point geometric crossover, in Parallel Problem Solving from Nature, PPSN XI, Lecture Notes in Computer Science, vol. 6238 (Springer, Berlin, 2010), pp. 83–93

  20. A. Moraglio, R. Poli, Topological interpretation of crossover. GECCO 1, 1377–1388 (2004)

    Google Scholar 

  21. N. Paterson, Genetic Programming with Context-Sensitive Grammars. Ph.D. thesis (School of Computer Science, University of St Andrews, Scotland, 2002)

  22. R. Poli, W. Langdon, N. McPhee, A Field Guide to Genetic Programming (Lulu Enterprises, UK Ltd, 2008)

  23. J. Quinlan, C4.5: Programs for Machine Learning (Morgan Kaufmann, San Francisco, 1993)

    Google Scholar 

  24. J. Quinlan, Decision trees and instance-based classifiers, in The Computer Science and Engineering Handbook (1996), pp. 521–535

  25. K. Rasheed, H. Hirsh, Informed operators: speeding up genetic-algorithm-based design optimization using reduced models, in Proceedings of the Genetic and Evolutionary Computation Conference (GECCO) (Morgan Kaufmann, Los Altos, CA, 2000), pp. 628–635

  26. W. Spears, Simple subpopulation schemes, in Proceedings of the Evolutionary Programming Conference (World Scientific, Singapore, 1994), pp. 296–307

  27. M. Šprogar, P. Kokol, Š.H. Babič, V. Podgorelec, M. Zorman, Vector decision trees. Intell. Data Anal. 4, 305–321 (2000)

    MATH  Google Scholar 

  28. W. Tackett, Recombination, selection, and the genetic construction of computer programs. Ph.D. thesis, (University of Southern California, Department of Electrical Engineering Systems, 1994)

  29. A. Teller, M. Veloso, Pado: Learning Tree Structured Algorithms for Orchestration into an Object Recognition System. Technical Report (Department of Computer Science, Carnegie Mellon University, Pittsburgh, USA, 1995)

  30. D. Wolpert, W. Macready, No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering and Computer Science, University of Maribor, Smetanova ul. 17, 2000, Maribor, Slovenia

    Matej Šprogar

Authors
  1. Matej Šprogar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matej Šprogar.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Šprogar, M. Prudent alignment and crossover of decision trees in genetic programming. Genet Program Evolvable Mach 16, 499–530 (2015). https://doi.org/10.1007/s10710-015-9243-7

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10710-015-9243-7

Keywords

Search

Navigation