Abstract
The proper management of diversity is essential to the success of Evolutionary Algorithms. Specifically, methods that explicitly relate the amount of diversity maintained in the population to the stopping criterion and elapsed period of execution, with the aim of attaining a gradual shift from exploration to exploitation, have been particularly successful. However, in the area of Genetic Programming, the performance of this design principle has not been studied. In this paper, a novel Genetic Programming method, Genetic Programming with Dynamic Management of Diversity (GP-DMD), is presented. GP-DMD applies this design principle through a replacement strategy that combines penalties based on distance-like functions with a multi-objective Pareto selection based on accuracy and simplicity. The proposed general method was adapted to the well-established Symbolic Regression benchmark problem using tree-based Genetic Programming. Several state-of-the-art diversity management approaches were considered for the experimental validation, and the results obtained showcase the improvements both in terms of mean square error and size. The effects of GP-DMD on the dynamics of the population are also analyzed, revealing the reasons for its superiority. As in other fields of Evolutionary Computation, this design principle contributes significantly to the area of Genetic Programming.
Similar content being viewed by others
Notes
The selected components are straightforward and commonly used and, although different choices are possible, the core idea is that all algorithms share the same selection in most components.
Note that this decision is probably the most difficult one when adapting GP-DMD to other applications. In the experimental validation, some results with alternative distance-like functions are also presented.
These instances belong to different benchmark sets, so they are quite different, and their small sample sizes allow fast runs, meaning that these plots can be easily used in the future for comparison purposes.
References
J.R. Koza, J.R. Koza, Genetic programming: on the programming of computers by means of natural selection, volume 1. MIT press, (1992)
R. Poli, J. Koza, Genetic programming, in Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques. ed. by E.K. Burke, G. Kendall (Springer, Boston, 2014), pp. 143–185
K. Nag, N.R. Pal, Genetic programming for classification and feature selection. in Evolutionary and swarm intelligence algorithms, Springer International Publishing, Cham, (2019), pp. 119–141
K. Krawiec, Behavioral Program Synthesis with Genetic Programming, vol. 618 (Springer, 2016)
I. Azaria, A. Elyasaf, M. Sipper, Evolving artificial general intelligence for video game controllers, in Genetic Programming Theory and Practice XIV. ed. by R. Riolo, B. Worzel, B. Goldman, B. Tozier (Springer International Publishing, Cham, 2018), pp. 53–63
C. Segura, A.H. Aguirre, S.I.V. Pena, S.B. Rionda, The importance of proper diversity management in evolutionary algorithms for combinatorial optimization, in NEO 2015: Results of the Numerical and Evolutionary Optimization Workshop NEO 2015 held at September 23–25 2015 in Tijuana Mexico. ed. by O. Schutze, L. Trujillo, P. Legrand, Y. Maldonado (Springer International Publishing, Cham, 2017), pp. 121–148
M. Crepinsek, S.-H. Liu, M. Mernik, Exploration and exploitation in evolutionary algorithms: a survey. ACM Comput. Surv. 45(3), 1–33 (2013)
C. Segura, A. Hernández-Aguirre, F. Luna, E. Alba, Improving diversity in evolutionary algorithms: new best solutions for frequency assignment. IEEE Trans. Evol. Comput. 21(4), 539–553 (2017)
C. Segura, S.B. Rionda, A.H. Aguirre, S.I.V. Pena, A novel diversity-based evolutionary algorithm for the traveling salesman problem. in Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO ’15, Association for Computing Machinery, New York, NY, USA, (2015). pp. 489–496
J.C. Castillo, C. Segura, Differential evolution with enhanced diversity maintenance. Opt. Lett. 14(6), 1471–1490 (2020)
J. Chacon, C. Segura, Analysis and enhancement of simulated binary crossover. in 2018 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8, (2018)
E. Burke, S. Gustafson, G. Kendall, N. Krasnogor, Advanced population diversity measures in genetic programming, in Parallel Problem Solving from Nature: PPSN VII. ed. by J.J.M. Guervos, P. Adamidis, H.-G. Beyer, H.-P. Schwefel, J.-L. Fernandez-Villacan (Springer, Berlin, Heidelberg, 2002), pp. 341–350
G. Squillero, A. Tonda, Divergence of character and premature convergence: a survey of methodologies for promoting diversity in evolutionary optimization. Inf. Sci. 329, 782–799 (2016). (Special issue on Discovery ScienceSpecial issue on Discovery Science)
N.T. Hien, N.X. Hoai, A brief overview of population diversity measures in genetic programming. in Proc. 3rd Asian-Pacific Workshop on Genetic Programming, Hanoi, Vietnam, pp. 128–139. Citeseer, (2006)
A.R. Burks, W.F. Punch, An efficient structural diversity technique for genetic programming. in Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO ’15, New York, NY, USA, (2015). Association for Computing Machinery, page 991–998
E.K. Burke, S. Gustafson, G. Kendall, Diversity in genetic programming: an analysis of measures and correlation with fitness. IEEE Trans. Evol. Comput. 8(1), 47–62 (2004)
U.-M. OReilly, J. Kelly, E. Hemberg, Improving genetic programming with novel exploration: exploitation control. Eur. Conf. Genet. Progr. 11451, 64–80 (2019)
M. Keijzer, Efficiently Representing Populations in Genetic Programming (MIT Press, Cambridge, 1996), pp. 259–278
M. Gaudesi, G. Squillero, A. Tonda, Universal information distance for genetic programming. in Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation, GECCO Comp ’14, New York, NY, USA, 2014. Association for Computing Machinery, pp. 137–138
L. Vanneschi, M. Castelli, S. Silva, A survey of semantic methods in genetic programming. Genet. Progr. Evol. Mach. 15(2), 195–214 (2014)
G. Folino, C. Pizzuti, G. Spezzano, L. Vanneschi, M. Tomassin, Diversity analysis in cellular and multipopulation genetic programming. In The 2003 Congress on Evolutionary Computation, 2003. CEC ’03, (2003) vol. 1, pp. 305–311
N.Q. Uy, N.X. Hoai, M. ONeill, R.I. McKay, E. Galvan-Lopez, Semantically-based crossover in genetic programming: application to real-valued symbolic regression. Genet. Progr. Evol. Mach. 12(2), 91–119 (2011)
Q.U. Nguyen, X.H. Nguyen, M. Oneill, A. Agapitos, An investigation of fitness sharing with semantic and syntactic distance metrics. in European Conference on Genetic Programming, Springer, (2012), Vol. 7244, pp. 109–120
J.P. Rosca, Entropy-driven adaptive representation. in Proceedings of the workshop on genetic programming: From theory to real-world applications, vol. 9, Tahoe City, California, USA, (1995). Citeseer
W. La Cava, L. Spector, and K. Danai, Epsilon-lexicase selection for regression. in Proceedings of the Genetic and Evolutionary Computation Conference 2016, GECCO ’16, New York, NY, USA, (2016). Association for Computing Machinery, pp. 741–748
B. Metevier, A.K. Saini, L. Spector, Lexicase Selection Beyond Genetic Programming (Springer International Publishing, Cham, 2019), pp. 123–136
E. Galvan-Lopez, B. Cody-Kenny, L. Trujillo, A. Kattan, Using semantics in the selection mechanism in genetic programming: a simple method for promoting semantic diversity. in 2013 IEEE Congress on Evolutionary Computation, (2013), pp. 2972–2979
P. Day, A.K. Nandi, Binary string fitness characterization and comparative partner selection in genetic programming. IEEE Trans. Evol. Comput. 12(6), 724–735 (2008)
A.K. Nandi, M.W. Aslam, Z. Zhu, Diverse partner selection with brood recombination in genetic programming. Appl. Soft Comput. 67, 558–566 (2018)
E.D. de Jong, R.A. Watson, J.B. Pollack, Reducing bloat and promoting diversity using multi-objective methods. in Proceedings of the 3rd Annual Conference on Genetic and Evolutionary Computation, GECCO’01, San Francisco, CA, USA, (2001). Morgan Kaufmann Publishers Inc, pp. 11–18
M. Schmidt, H. Lipson, Age-Fitness Pareto Optimization (Springer, New York, 2011), pp. 129–146
J. Grefenstette, Genetic algorithms for changing environments. in Parallel Problem Solving from Nature 2, Elsevier, (1992), pp. 137–144
A.R. Burks, W.F. Punch, An analysis of the genetic marker diversity algorithm for genetic programming. Genet. Progr. Evol. Mach. 18(2), 213–245 (2017)
N.F. McPhee, N.J. Hopper, Analysis of genetic diversity through population history. in Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation - Volume 2, GECCO’99, San Francisco, CA, USA, (1999). Morgan Kaufmann Publishers Inc, pp. 1112–1120
B. Cao, Z. Jiang, Increasing diversity and controlling bloat in linear genetic programming. in 2016 3rd International Conference on Information Science and Control Engineering (ICISCE), (2016), pp. 414–419
G.S. Hornby, A Steady-State Version of the Age-Layered Population Structure EA (Springer, US, Boston, MA, 2010), pp. 87–102
A.R. Burks, W.F. Punch, An investigation of hybrid structural and behavioral diversity methods in genetic programming, in Genetic Programming Theory and Practice XIV. ed. by R. Riolo, B. Worzel, B. Goldman, B. Tozier (Springer International Publishing, Cham, 2018), pp. 19–34
E.R. Ruiz, C. Segura, Memetic algorithm with Hungarian matching based crossover and diversity preservation. Comput. Sist. 22(2), 07 (2018)
D.R. White, J. Mdermott, M. Castelli, L. Manzoni, B.W. Goldman, G. Kronberger, W. Jaskowski, U.-M. OReilly, S. Luke, Better GP benchmarks: community survey results and proposals. Genet. Progr. Evol. Mach. 14(1), 3–29 (2013)
P. Orzechowski, W La Cava, J.H. Moore, Where are we now? a large benchmark study of recent symbolic regression methods. In Proceedings of the Genetic and Evolutionary Computation Conference, GECCO ’18, New York, NY, USA, (2018). Association for Computing Machinery, pp. 1183–1190
J. Zegklitz and P. Posik, Benchmarking state-of-the-art symbolic regression alrogirhms. Genet. Progr. Evol. Mach., pp. 1–29, (2020)
I. Arnaldo, K. Krawiec, U-M OReilly, Multiple regression genetic programming. in Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, GECCO ’14, New York, NY, USA, (2014). Association for Computing Machinery, pp. 879–886
J. McDermott, D.R. White, S. Luke, L. Manzoni, M. Castelli, L. Vanneschi, W. Jaskowski, K. Krawiec, R. Harper, K. De Jong, and U-M OReilly, Genetic programming needs better benchmarks. In Proceedings of the 14th Annual Conference on Genetic and Evolutionary Computation, GECCO ’12, New York, NY, USA, (2012). ACM, pp. 791–798
M. Virgolin, A. De Lorenzo, E. Medvet, F. Randone, Learning a formula of interpretability to learn interpretable formulas, in Parallel Problem Solving from Nature: PPSN XVI. ed. by T. Back, M. Preuss, A. Deutz, H. Wang, C. Doerr, M. Emmerich, H. Trautmann (Springer International Publishing, Cham, 2020), pp. 79–93
M. Virgolin, A. De Lorenzo, F. Randone, E. Medvet, M. Wahde, Model learning with personalized interpretability estimation (ml-pie). In Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO ’21, New York, NY, USA, (2021). Association for Computing Machinery, pp. 1355–1364
E.D. De Jong, J.B. Pollack, Multi-objective methods for tree size control. Genet. Progr. Evol. Mach. 4(3), 211–233 (2003)
E.P. Bezier, S. Sioussiou, Semi-automatic system for defining free-form curves and surfaces. Comput.-Aid. Des. 15(2), 65–72 (1983)
Acknowledgements
Authors acknowledge the financial support from CONACyT through the “Ciencia Básica” project no. 285599 and the support from “Laboratorio de Supercómputo del Bajio” through the project 300832 from CONACyT.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Nieto-Fuentes, R., Segura, C. GP-DMD: a genetic programming variant with dynamic management of diversity. Genet Program Evolvable Mach 23, 279–304 (2022). https://doi.org/10.1007/s10710-021-09426-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10710-021-09426-4