Skip to main content
SpringerLink
Log in

Geometric semantic genetic programming with normalized and standardized random programs

  • Published:
Genetic Programming and Evolvable Machines Aims and scope Submit manuscript

Abstract

Geometric semantic genetic programming (GSGP) represents one of the most promising developments in the area of evolutionary computation (EC) in the last decade. The results achieved by incorporating semantic awareness in the evolutionary process demonstrate the impact that geometric semantic operators have brought to the field of EC. An improvement to the geometric semantic mutation (GSM) operator is proposed, inspired by the results achieved by batch normalization in deep learning. While, in one of its most used versions, GSM relies on the use of the sigmoid function to constrain the semantics of two random programs responsible for perturbing the parent’s semantics, here a different approach is followed, which allows reducing the size of the resulting programs and overcoming the issues associated with the use of the sigmoid function, as commonly done in deep learning. The idea is to consider a single random program and use it to perturb the parent’s semantics only after standardization or normalization. The experimental results demonstrate the suitability of the proposed approach: despite its simplicity, the presented GSM variants outperform standard GSGP on the studied benchmarks, with a difference in terms of performance that is statistically significant. Furthermore, the individuals generated by the new GSM variants are easier to simplify, allowing us to create accurate but significantly smaller solutions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Only the definitions of GSOs for symbolic regression problems are presented in this paper as they are the only ones used. For the definitions of GSOs in other domains, the reader is referred to [28].

  2. Given the nature of GSM and GSC, the ideas discussed here generalize to any type of program representation (trees, linear representations, graphs, etc.). The experimental work will use a linear genome representation for programs.

  3. Version 1.10.1: https://github.com/sympy/sympy.

  4. Boxplots are used instead of violin plots for easier depiction.

References

  1. J. Albinati, G.l. Pappa, F.E. Otero et al., The effect of distinct geometric semantic crossover operators in regression problems, in Genetic Programming: 18th European Conference, EuroGP 2015, Copenhagen, Denmark, April 8–10, 2015, Proceedings 18 (Springer, 2015), pp. 3–15

  2. I. Bakurov, L. Vanneschi, M. Castelli et al., Edda-v2—An improvement of the evolutionary demes despeciation algorithm, in Parallel Problem Solving from Nature—PPSN XV. ed. by A. Auger, C.M. Fonseca, N. Lourenço et al. (Springer International Publishing, Cham, 2018), pp. 185–196

  3. I. Bakurov, M. Buzzelli, M. Castelli et al., General purpose optimization library (GPOL): a flexible and efficient multi-purpose optimization library in python. Appl. Sci. (2021). https://doi.org/10.3390/app11114774

    Article  Google Scholar 

  4. I. Bakurov, M. Castelli, O. Gau et al., Genetic programming for stacked generalization. Swarm Evol. Comput. 65, 100913 (2021). https://doi.org/10.1016/j.swevo.2021.100913

    Article  Google Scholar 

  5. I. Bakurov, M. Castelli, F. Fontanella et al., A novel binary classification approach based on geometric semantic genetic programming. Swarm Evol. Comput. 69, 101028 (2022). https://doi.org/10.1016/j.swevo.2021.101028

    Article  Google Scholar 

  6. L. Beadle, C.G. Johnson, Semantically driven crossover in genetic programming, in 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence) (IEEE, 2008), pp. 111–116

  7. L. Beadle, C.G. Johnson, Semantic analysis of program initialisation in genetic programming. Genet. Program Evolvable Mach. 10, 307–337 (2009)

    Article  Google Scholar 

  8. L. Beadle, C.G. Johnson, Semantically driven mutation in genetic programming, in 2009 IEEE Congress on Evolutionary Computation (IEEE, 2009), pp. 1336–1342

  9. M. Castelli, S. Silva, L. Vanneschi, A C++ framework for geometric semantic genetic programming. Genet. Program Evolvable Mach. 16(1), 73–81 (2015)

    Article  Google Scholar 

  10. M. Castelli, L. Trujillo, L. Vanneschi et al., Geometric semantic genetic programming with local search, in Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, GECCO ’15 (Association for Computing Machinery, New York, NY, USA, 2015), pp. 999–1006. https://doi.org/10.1145/2739480.2754795

  11. M. Castelli, L. Manzoni, I. Gonçalves, et al., An analysis of geometric semantic crossover: a computational geometry approach, in International Joint Conference on Computational Intelligence (2016)

  12. F. Chollet et al., Keras (2015). https://keras.io

  13. J. Derrac, S. García, D. Molina et al., A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evol. Comput. 1(1), 3–18 (2011). https://doi.org/10.1016/j.swevo.2011.02.002

    Article  Google Scholar 

  14. I. Gonçalves, S. Silva, C.M. Fonseca, On the generalization ability of geometric semantic genetic programming, in Genetic Programming. ed. by P. Machado, M.I. Heywood, J. McDermott et al. (Springer, Cham, 2015), pp.41–52

    Chapter  Google Scholar 

  15. I. Gonçalves, S. Silva, C.M. Fonseca, Semantic learning machine: A feedforward neural network construction algorithm inspired by geometric semantic genetic programming, in Progress in Artificial Intelligence. ed. by F. Pereira, P. Machado, E. Costa et al. (Springer, Cham, 2015), pp.280–285

    Chapter  Google Scholar 

  16. I. Gonçalves, S. Silva, C.M. Fonseca et al., Unsure when to stop?, in Proceedings of the Genetic and Evolutionary Computation Conference (ACM, 2017). https://doi.org/10.1145/3071178.3071328

  17. I. Gonçalves, An exploration of generalization and overfitting in genetic programming: standard and geometric semantic approaches. Ph.D. Thesis, Department of Informatics Engineering, University of Coimbra, Portugal., Coimbra, Portugal (2017), available at https://www.cisuc.uc.pt/download-file/13946/sfxgEyeIRXv2dxxWgZS5

  18. I.J. Goodfellow, Y. Bengio, A. Courville, Deep Learning (MIT Press, Cambridge, 2016)

    Google Scholar 

  19. K. He, Z. Zhang, S. Ren et al., Deep residual learning for image recognition, in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), pp. 770–778. https://doi.org/10.1109/CVPR.2016.90

  20. G. Huang, Z. Liu, L.V.D. Maaten et al., Densely connected convolutional networks, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE Computer Society, Los Alamitos, CA, USA, 2017), pp. 2261–2269. https://doi.org/10.1109/CVPR.2017.243, https://doi.ieeecomputersociety.org/10.1109/CVPR.2017.243

  21. S. Ioffe, C. Szegedy, Batch normalization: Accelerating deep network training by reducing internal covariate shift, in Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37. JMLR.org, ICML’15 (2015) pp. 448–456

  22. J.R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection, vol. 1 (MIT Press, Cambridge, 1992)

    Google Scholar 

  23. J.R. Koza, Human-competitive results produced by genetic programming. Genet. Program Evolvable Mach. 11, 251–284 (2010)

    Article  Google Scholar 

  24. Y.A. LeCun, L. Bottou, G.B. Orr et al., Efficient BackProp (Springer, Berlin, 2012), pp.9–48. https://doi.org/10.1007/978-3-642-35289-8_3

    Book  Google Scholar 

  25. J.F.B.S. Martins, L.O.V.B. Oliveira, L.F. Miranda et al., Solving the exponential growth of symbolic regression trees in geometric semantic genetic programming, in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO ’18 (ACM, New York, NY, USA, 2018), pp. 1151–1158

  26. J. McDermott, A. Agapitos, A. Brabazon et al., Geometric semantic genetic programming for financial data, in Applications of Evolutionary Computation: 17th European Conference, EvoApplications 2014, Granada, Spain, April 23–25, 2014, Revised Selected Papers 17, (Springer, 2014), pp. 215–226

  27. NF. McPhee, B. Ohs, T. Hutchison, Semantic building blocks in genetic programming, in Genetic Programming: 11th European Conference, EuroGP 2008, Naples, Italy, March 26–28, 2008. Proceedings 11 (Springer, 2008), pp. 134–145

  28. A. Moraglio, K. Krawiec, C. Johnson, Geometric semantic genetic programming, in Parallel Problem Solving from Nature—PPSN XII, ed. by C. Coello, V. Cutello, K. Deb, et al. Lecture Notes in Computer Science, vol. 7491 (Springer, Berlin, 2012), pp. 21–31

  29. V. Nair, G.E. Hinton, Rectified linear units improve restricted Boltzmann machines, in Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML’10 (Omnipress, Madison, WI, USA, 2010), pp. 807–814

  30. M. Nicolau, J. McDermott, Genetic programming symbolic regression: What is the prior on the prediction?, in Genetic Programming Theory and Practice XVII (2020), pp. 201–225

  31. L.O.V. Oliveira, F.E. Otero, G.L. Pappa, A dispersion operator for geometric semantic genetic programming, in Proceedings of the Genetic and Evolutionary Computation Conference, vol. 2016 (2016), pp. 773–780

  32. I. Ortigosa, J.G.R. Lopez, A neural networks approach to residuary resistance of sailing yachts prediction, in Proceedings of the International Conference on Marine Engineering MARINE (2007), p. 250

  33. A. Paszke, S. Gross, F. Massa et al., PyTorch: An Imperative Style, High-Performance Deep Learning Library (Curran Associates Inc., Red Hook, NY, USA, 2019)

    Google Scholar 

  34. J.R. Quinlan, Combining instance-based and model-based learning, in Machine Learning, Proceedings of the Tenth International Conference, University of Massachusetts, Amherst, MA, USA, June 27–29, 1993 (1993), pp. 236–243

  35. L. Trujillo, J.M. Muñoz Contreras, D.E. Hernandez et al., GSGP-CUDA—a CUDA framework for geometric semantic genetic programming. SoftwareX 18, 101085 (2022). https://doi.org/10.1016/j.softx.2022.101085

    Article  Google Scholar 

  36. A. Tsanas, A. Xifara, Accurate quantitative estimation of energy performance of residential buildings using statistical machine learning tools. Energy Build. 49, 560–567 (2012)

    Article  Google Scholar 

  37. N.Q. Uy, N.X. Hoai, M. O’Neill et al., Semantically-based crossover in genetic programming: application to real-valued symbolic regression. Genet. Program Evolvable Mach. 12, 91–119 (2011)

    Article  Google Scholar 

  38. L. Vanneschi, S. Silva, M. Castelli et al., Geometric semantic genetic programming for real life applications, in Genetic Programming Theory and Practice XI (2014), pp. 191–209

  39. L. Vanneschi, S. Silva, M. Castelli et al., Geometric Semantic Genetic Programming for Real Life Applications (Springer, New York, 2014), pp.191–209

    Google Scholar 

  40. L. Vanneschi, I. Bakurov, M. Castelli, An initialization technique for geometric semantic GP based on demes evolution and despeciation, in 2017 IEEE Congress on Evolutionary Computation (CEC) (2017), pp. 113–120. https://doi.org/10.1109/CEC.2017.7969303

  41. E.J. Vladislavleva, G.F. Smits, D. den Hertog, Order of nonlinearity as a complexity measure for models generated by symbolic regression via pareto genetic programming. IEEE Trans. Evol. Comput. 13(2), 333–349 (2009)

    Article  Google Scholar 

  42. I.C. Yeh, Modeling of strength of high-performance concrete using artificial neural networks. Cem. Concr. Res. 28(12), 1797–1808 (1998)

    Article  CAS  Google Scholar 

Download references

Acknowledgements

This work was partially supported by FCT, Portugal, through funding of research units MagIC/NOVA IMS (UIDB/04152/2020) and LASIGE (UIDB/00408/2020 and UIDP/00408/2020). This work also was supported by CONACYT (Mexico) Project CF-2023-I-724, TecNM (Mexico) Project 16788.23-P and Project 17756.23-P. José Manuel Muñoz Contreras was supported by CONACYT scholarship 771416; Nuno Rodrigues was supported by FCT PhD Grant 2021/05322/BD.

Author information

Author notes

  1. Illya Bakurov and José Manuel Muñoz Contreras have contributed equally to this work.

Authors and Affiliations

  1. NOVA Information Management School (NOVA IMS), Universidade Nova de Lisboa, Campus de Campolide, 1070-312, Lisbon, Portugal

    Illya Bakurov, Mauro Castelli & Leonardo Vanneschi

  2. Departamento de Ingeniería Eléctrica y Electrónica, Tecnológico Nacional de México/IT de Tijuana, 22500, Tijuana, BC, Mexico

    José Manuel Muñoz Contreras & Leonardo Trujillo

  3. LASIGE, Department of Informatics, Faculty of Sciences, University of Lisbon, Lisbon, Portugal

    Nuno Rodrigues, Sara Silva & Leonardo Trujillo

  4. Department of Computer Science & Engineering and Beacon Center for the Study of Evolution in Action, Michigan State University, East Lansing, MI, 48824, USA

    Illya Bakurov

Authors
  1. Illya Bakurov
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. José Manuel Muñoz Contreras
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mauro Castelli
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nuno Rodrigues
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sara Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Leonardo Trujillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Leonardo Vanneschi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conceptualization: IB, MC, NR, SS, LV; methodology: IB, JMMC, LT, LV; formal analysis and investigation: IB, JMMC, LT; writing—original draft preparation: IB, JMMC, MC, NR, LV; writing—review and editing: MC, SS, LT, LV; funding acquisition: LT, LV, MC, SS; supervision: LT, LV.

Corresponding author

Correspondence to Leonardo Trujillo.

Ethics declarations

Conflict of interest

The authors have no conflicts of interest to declare that are relevant to the content of this article. Several authors are board members of genetic programming and evolvable machines. Leonardo Trujillo, Sara Silva and Leonardo Vanneschi are Associate Editors, while Mauro Castelli is on the Editorial Board.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bakurov, I., Muñoz Contreras, J.M., Castelli, M. et al. Geometric semantic genetic programming with normalized and standardized random programs. Genet Program Evolvable Mach 25, 6 (2024). https://doi.org/10.1007/s10710-024-09479-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10710-024-09479-1

Keywords

Search

Navigation